-- Logs begin at Mon 2022-08-01 14:23:30 UTC, end at Mon 2022-08-01 15:32:54 UTC. -- Aug 01 14:56:32.304873 dv-cinder-803823-14-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 01 14:56:32.314800 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 01 14:56:32.317271 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 1 14:56:32 2022] *** Aug 01 14:56:32.317271 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 01 14:56:32.317271 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 01 14:56:32.317271 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: nodename: dv-cinder-803823-14-iscsi Aug 01 14:56:32.317271 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: machine: x86_64 Aug 01 14:56:32.317271 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: clock source: unix Aug 01 14:56:32.317271 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: pcre jit disabled Aug 01 14:56:32.317271 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: detected number of CPU cores: 4 Aug 01 14:56:32.317271 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: current working directory: / Aug 01 14:56:32.317271 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: detected binary path: /usr/bin/uwsgi-core Aug 01 14:56:32.317271 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: your processes number limit is 47680 Aug 01 14:56:32.317271 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: your memory page size is 4096 bytes Aug 01 14:56:32.317271 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: detected max file descriptor number: 2048 Aug 01 14:56:32.317271 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: lock engine: pthread robust mutexes Aug 01 14:56:32.318617 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: thunder lock: enabled Aug 01 14:56:32.318617 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 01 14:56:32.318617 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 01 14:56:32.359722 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: Python main interpreter initialized at 0x561b711919c0 Aug 01 14:56:32.359722 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: python threads support enabled Aug 01 14:56:32.359722 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: your server socket listen backlog is limited to 100 connections Aug 01 14:56:32.359722 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: your mercy for graceful operations on workers is 90 seconds Aug 01 14:56:32.360311 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: mapped 403077 bytes (393 KB) for 2 cores Aug 01 14:56:32.360311 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: *** Operational MODE: preforking *** Aug 01 14:56:32.360553 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: *** uWSGI is running in multiple interpreter mode *** Aug 01 14:56:32.360553 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: spawned uWSGI master process (pid: 80400) Aug 01 14:56:32.360827 dv-cinder-803823-14-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 01 14:56:32.363003 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: spawned uWSGI worker 1 (pid: 80411, cores: 1) Aug 01 14:56:32.363003 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: spawned uWSGI worker 2 (pid: 80412, cores: 1) Aug 01 14:56:32.363003 dv-cinder-803823-14-iscsi devstack@keystone.service[80400]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 01 14:56:38.388495 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 14:56:38.388495 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: from cryptography.utils import int_from_bytes Aug 01 14:56:38.389721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 14:56:38.389721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: from cryptography.utils import int_from_bytes Aug 01 14:56:38.445875 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 14:56:38.446140 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 14:56:38.446798 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] command line args: [] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 14:56:38.449212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 14:56:38.449212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ================================================================================ {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 14:56:38.449212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] admin_token = **** {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.449212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] config_dir = [] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.449212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.449212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] config_source = [] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.449212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] debug = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.449877 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.449877 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.449877 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.449877 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.449877 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.450615 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] list_limit = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.450615 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] log_config_append = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.450615 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.450615 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] log_dir = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.450977 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] log_file = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.451081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.451361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.453944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.453944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.453944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.453944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.453944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.453944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.453944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.454937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.454937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.454937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.454937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.454937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.454937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.455729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.93/identity {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.455729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] publish_errors = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.455729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.456100 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.456100 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.456369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.456584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.456858 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] standard_threads = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.457119 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.457388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.457593 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.457799 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] use_journal = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.457989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] use_json = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.458177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] use_stderr = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.458361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] use_syslog = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.458667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.458907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.459177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.459478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.459769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.460004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.460259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.460530 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.460798 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.461073 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.461314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 14:56:38.461314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: from cryptography.utils import int_from_bytes Aug 01 14:56:38.461942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.461942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.462705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 14:56:38.462705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: from cryptography.utils import int_from_bytes Aug 01 14:56:38.462842 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.462842 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.462953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.463197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.463460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.463701 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.463963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.464222 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.464486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.464738 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.464998 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.465297 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.465574 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.465852 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.466113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.466372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.466626 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.466902 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.467196 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.467451 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.467708 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.467965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.468254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.468664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.468864 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.469078 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.469329 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.469569 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.469936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.470056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.470259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.470493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.470728 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.470990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.471357 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.471584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.471804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.472029 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.472286 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.472573 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.472835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.473069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.473326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.473589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.473835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.474078 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.474319 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.474553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.474808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.475049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.475291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.475548 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.475836 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.476118 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.476381 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.476669 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.476914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.477239 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.477536 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.477812 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.478091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.478375 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.478653 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.478995 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.479299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.479681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.479841 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.480242 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.480583 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.480912 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.481172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.481439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.481757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.482039 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.482291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.482541 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.482788 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.483034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.483316 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.483562 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.483805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.484043 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.484277 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.484546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.484778 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.485012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.485260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.485505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.485761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.486001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.486263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.486530 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.486773 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.487013 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] auth.external = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.487256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.487556 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.487812 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.488070 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] auth.password = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.488333 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] auth.token = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.488630 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.488873 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.489127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.489361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.489633 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.489914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.490148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.490391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] credential.caching = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.490642 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.490899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.491175 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.491431 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.491698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.491970 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.492250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.492553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.492783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.493034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.493983 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.494256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.494517 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.494790 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.495083 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.495322 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] federation.caching = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.495634 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.495895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.496230 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.496450 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.496830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.496968 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.497127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.497448 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.497913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.498234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.498434 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.498679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] identity.caching = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.499223 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.499223 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.499762 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.499762 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.500084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.500244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.500669 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.500836 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.501070 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.501337 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.501596 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.501836 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.502090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.502462 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.502631 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.502975 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.503288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.503738 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.503962 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.504396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.504880 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.505312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.505777 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.506192 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.506636 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.507081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.507567 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.507991 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.508435 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.508754 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.509548 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.509826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.510120 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.510369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.510640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.510876 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.511122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.511400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.511679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.511970 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.512206 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.512489 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.512755 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.512993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.513230 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.513483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.513734 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.513981 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.514223 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.514483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.514574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 14:56:38.514774 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 14:56:38.514922 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.user = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.515082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] command line args: [] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 14:56:38.515348 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.515442 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 14:56:38.515517 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.515589 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ================================================================================ {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 14:56:38.515875 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.515996 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] admin_token = **** {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.516070 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.516392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] config_dir = [] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.516575 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.516724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.516909 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.517084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] config_source = [] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.517084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] debug = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.517269 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.517393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.517546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.517731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.519618 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.519926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.520049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.520267 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.520506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.520628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.520699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.520910 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] list_limit = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.520990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.521061 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] log_config_append = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.521061 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.521448 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.521582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] log_dir = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.521725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.521819 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] log_file = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.521819 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.522207 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.522337 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.522458 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.522586 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.522710 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.522825 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.522958 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.523071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.523361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.523495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.523621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.523712 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.523811 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.523887 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.523966 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.524072 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.524194 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.524291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.524518 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.524653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.524653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.524862 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.524937 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.525012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.525112 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.525177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.525295 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.525416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.525416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.525416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.525769 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.93/identity {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.525769 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] publish_errors = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.526016 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.526080 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.526159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.526251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.526325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.526386 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.526446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.526504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.526563 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.526630 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.526691 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] resource.caching = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.526750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] standard_threads = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.526868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.526986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.527062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.527169 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.527292 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.527374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.527374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] use_journal = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.527515 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.527648 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] use_json = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.527785 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.527916 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] use_stderr = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.528047 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.528173 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] use_syslog = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.528291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.528399 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:56:38.528568 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.528695 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.528780 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.528842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.528901 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] role.caching = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.528960 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.529029 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] role.driver = sql {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.529136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.529229 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.529348 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.529540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.529675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.529809 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.529893 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.529954 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.530030 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.530138 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.530266 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.530405 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.530489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.530572 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.530671 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.530750 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.530832 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.530952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.531084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.531208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.531357 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.531452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.531563 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.531713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.531803 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.531867 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.531949 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.532063 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.532141 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.532141 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.532343 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.532434 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.532560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.532634 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.532707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.532793 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.532888 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.532952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.533012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.533086 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.533170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.533249 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.533311 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.533370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.533449 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.533515 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.533652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.533728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.533789 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.533856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.533937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.533999 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.534087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.534216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.534349 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.534467 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.534552 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.534651 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.534723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.534797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.534877 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.535009 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.535107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.535196 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.535287 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.535388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.535450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.535602 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] token.caching = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.535773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.535909 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.536059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.536191 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.536327 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.536509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.536650 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.536650 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.536913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.537049 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.537178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.537305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.537437 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.537567 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.537705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.537874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.538006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.538134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.538262 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.538390 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.538539 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.538673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.538804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.538937 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.539071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.539202 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.539347 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.539481 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.539611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.539744 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.539875 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.540039 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.540184 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.540317 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.540449 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.540618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.540751 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.connection = **** {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.540902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.541044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.541184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.541313 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.541449 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.541583 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.541710 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.541838 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.541981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.542124 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.542269 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.542403 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.542543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.542687 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.542834 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.542961 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.543088 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.543212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.543338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.543464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.543572 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.543572 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.543739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.543803 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.543872 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.543955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.544040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.544109 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.544170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.544230 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.544290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.544364 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.544446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.544555 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.544555 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.544670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.544760 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.544873 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.544873 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 14:56:38.545130 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.545130 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.545269 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.545513 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.545834 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.546116 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.546397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.546637 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.546903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.547211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.547424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.547694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.548023 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.548264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.548536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.548787 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.549018 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.549295 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.549539 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.549824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.550062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.550306 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.550548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.550857 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.551087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.551398 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.551665 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.551903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.552138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] auth.external = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.552436 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.552768 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.553000 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.553279 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] auth.password = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.553546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] auth.token = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.553862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.554114 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.554366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.554604 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.554920 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.555197 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.555506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.556294 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] credential.caching = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.556574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.556843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.557095 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.557339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.557586 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.557891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.558154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.558357 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.558661 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.558951 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.559256 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.559547 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.559819 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.560074 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.560335 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.560605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] federation.caching = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.560843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.561092 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.561339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.561575 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.561823 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.562073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.562342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.562660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.562918 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.563157 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.563496 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.563697 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] identity.caching = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.563994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.564224 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.564580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.564833 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.565078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.565390 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.565658 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.565963 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.566220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.566458 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.566694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.566927 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.567177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.567426 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.567674 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.568032 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.568322 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.568696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.568936 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.569184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.569498 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.569740 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.570014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.570267 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.570513 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.570758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.571082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.571969 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.572179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.572541 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.572684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.574907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.574907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.574907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.574907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.574907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.574907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.574907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.574907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.574907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.576405 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.576405 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.576405 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.576405 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.576405 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.576405 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.577505 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.577505 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.577505 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.577505 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.577858 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.user = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.578033 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.578204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.578465 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.578712 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.578969 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.579212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.579472 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.579785 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.580032 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.580273 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.580552 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.580789 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.581039 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.581287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.581536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.581780 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.581954 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.582041 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.582229 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.582362 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.582519 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.582651 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.582816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.583008 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.583220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.583411 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.583616 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.583855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.584182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.584415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.584634 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.584882 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.585158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.585413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.585810 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.586082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.586403 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.586644 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.586925 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.587196 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] resource.caching = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.587529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.587811 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.588046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.588406 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.588685 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.588918 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.589161 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.589457 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.589717 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.589973 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] role.caching = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.590223 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] role.driver = sql {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.590457 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.590711 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.590958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.591284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.591568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.591805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.592134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.592384 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.592681 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.592854 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.593005 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.593320 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.593576 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.593847 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.594098 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.594543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.594852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.595113 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.595382 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.595705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.595970 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.596202 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.596551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.596830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.597083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.597316 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.597616 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.597813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.598087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.598364 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.598618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.598863 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.599189 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.599459 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.599673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] token.caching = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.599914 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.600190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.600549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.600826 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.601081 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.601310 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.601670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.601892 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.602170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.602468 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.602728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.602987 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.603300 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.603564 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.603796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.604045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.604349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.604657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.connection = **** {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.604916 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.605242 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.605485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.605773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.605979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.606222 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.606535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.606827 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.607032 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.607272 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.607519 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.607757 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.608006 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.608244 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.608596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.608834 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.609078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.609338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:56:38.609552 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 14:56:38.626267 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.626516 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.638099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.638303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.638652 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.648059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.679561 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.679801 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.686549 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.720063 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.742410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.742655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.761349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.761639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.817135 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.817357 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.817573 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.817703 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.817837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.817968 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.818117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.818204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.818280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.818354 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.818489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.818707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.844800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.845036 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.845131 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.845291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.845429 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.845569 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.845655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.845727 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.845803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.845943 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.846067 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.846185 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.846272 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.846592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.858982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.859670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.873740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.873962 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.874180 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.874302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.874431 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.874584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.874708 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.874910 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.875031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.875152 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.875296 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.875413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.875413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.875650 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:56:38.893897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.894107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 14:56:38.894327 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.894420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.894544 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 14:56:38.894620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 14:56:38.894849 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.894959 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.895093 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 14:56:38.895226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.895359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.895494 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.895639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.895774 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.895910 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.896048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.906253 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.906542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.906702 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.906839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.906971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.907131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.907313 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.907740 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.914830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:38.915821 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:38.920558 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:38.920871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.921181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.921490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.921644 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:38.921784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.921990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.922097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.922188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.922297 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.922477 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.922598 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.922761 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.922913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.923034 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.923156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.923286 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.923404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.923528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.923795 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.924070 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.924333 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.948845 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.950552 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:38.950868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.950962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.952720 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:38.953015 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.955793 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:38.956090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.956410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.958089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:38.958370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.958657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.963628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:38.963922 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.965716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:38.966075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.968501 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:38.968804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.969132 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.969422 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.969715 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.970005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.970319 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.970615 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.971210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:38.971510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.971792 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.972074 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.972346 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.972656 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.972946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.973210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.988015 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.988396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.991027 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.991458 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.992442 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:38.996545 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:38.996714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:38.996877 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.001323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.001323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.002219 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.005575 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.006163 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.008926 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.009273 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.009581 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.009710 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.009987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.010177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.010519 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.013375 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.013767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.014147 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.014502 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.014916 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.015196 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.027822 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.028142 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.028416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.028703 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.033723 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.034020 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.034289 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.034552 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.036070 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.037787 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.038091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.041807 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.042093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.042422 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.042724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.042838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.043056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.044512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.049176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.049176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.049176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.049176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.049176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.049176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.055076 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.055363 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.055703 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.056532 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.056845 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.057124 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.057407 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.061431 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.061723 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.062029 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.062375 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.062662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.062940 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.063213 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.074049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.077241 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.079790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.080350 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.082920 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.083136 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.084850 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.085174 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.085481 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.086171 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.089183 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.090688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.090988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.091358 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.092391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.094066 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.097318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.097620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.097906 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.098185 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.098478 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.100183 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.103747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.104039 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.104334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.104640 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.107120 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.107452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.107735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.112975 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.113270 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.113546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.115151 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.115496 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.115762 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.116084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.116378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.116714 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.117127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.117361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.117776 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.118185 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.119055 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.119348 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.119641 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.119924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.121718 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:56:39.122005 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.122287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.122829 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.123109 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.123377 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.123650 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.123932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.124313 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.124617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.124899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.125171 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.125442 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.125699 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:56:39.155737 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x561b711919c0 pid: 80411 (default app) Aug 01 14:56:39.159251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb08219-e9ec-42ca-b6c2-5f365ccc10cb None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:39.159540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb08219-e9ec-42ca-b6c2-5f365ccc10cb None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:39.159850 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb08219-e9ec-42ca-b6c2-5f365ccc10cb None None] PATH_INFO: `/v3` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:39.161302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x561b711919c0 pid: 80412 (default app) Aug 01 14:56:39.164716 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1/1] 10.222.0.93 () {50 vars in 791 bytes} [Mon Aug 1 14:56:39 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:56:47.169333 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1a214-96bf-47a7-b9a7-ac963574c42f None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:47.169910 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1a214-96bf-47a7-b9a7-ac963574c42f None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:47.169910 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1a214-96bf-47a7-b9a7-ac963574c42f None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:47.174333 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1/2] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:56:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:56:47.179849 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aba213-176e-4cc2-ae14-9f5b5ead0688 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:47.180444 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aba213-176e-4cc2-ae14-9f5b5ead0688 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:47.180444 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aba213-176e-4cc2-ae14-9f5b5ead0688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:47.251363 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: INFO dbcounter [None req-44aba213-176e-4cc2-ae14-9f5b5ead0688 None None] Registered counter for database keystone Aug 01 14:56:47.271300 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG oslo_db.sqlalchemy.engines [None req-44aba213-176e-4cc2-ae14-9f5b5ead0688 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80411) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 14:56:47.278896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writer thread running {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 14:56:47.474762 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG passlib.handlers.bcrypt [None req-44aba213-176e-4cc2-ae14-9f5b5ead0688 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80411) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 14:56:47.475102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG passlib.handlers.bcrypt [None req-44aba213-176e-4cc2-ae14-9f5b5ead0688 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80411) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 14:56:47.533187 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-44aba213-176e-4cc2-ae14-9f5b5ead0688 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:56:47.534862 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-44aba213-176e-4cc2-ae14-9f5b5ead0688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:56:47.674473 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 2/3] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:56:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 496 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 14:56:47.809030 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1a214-96bf-47a7-b9a7-ac963574c42f None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:47.879056 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: INFO dbcounter [None req-d3f1a214-96bf-47a7-b9a7-ac963574c42f None None] Registered counter for database keystone Aug 01 14:56:47.901356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG oslo_db.sqlalchemy.engines [None req-d3f1a214-96bf-47a7-b9a7-ac963574c42f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80412) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 14:56:47.907934 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writer thread running {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 14:56:48.030817 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1a214-96bf-47a7-b9a7-ac963574c42f None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:48.035236 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9173a1-e122-4115-b55c-fe9f63926e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:48.035877 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9173a1-e122-4115-b55c-fe9f63926e8a admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:48.036156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9173a1-e122-4115-b55c-fe9f63926e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:48.036436 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9173a1-e122-4115-b55c-fe9f63926e8a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:48.045979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9173a1-e122-4115-b55c-fe9f63926e8a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:48.098045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6e9173a1-e122-4115-b55c-fe9f63926e8a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 14:56:48.098390 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9173a1-e122-4115-b55c-fe9f63926e8a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:48.109651 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 2/4] 10.222.0.93 () {58 vars in 1324 bytes} [Mon Aug 1 14:56:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:56:48.115674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aba213-176e-4cc2-ae14-9f5b5ead0688 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:48.131177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aba213-176e-4cc2-ae14-9f5b5ead0688 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:48.135690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32e370-7c24-456d-b26d-3d42ad494de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:48.136859 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32e370-7c24-456d-b26d-3d42ad494de0 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:48.137146 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32e370-7c24-456d-b26d-3d42ad494de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:48.137430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32e370-7c24-456d-b26d-3d42ad494de0 admin admin] PATH_INFO: `/v3/projects/28ee187de5744709bb65431943eb0200` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:48.138953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32e370-7c24-456d-b26d-3d42ad494de0 admin admin] RBAC: Authorizing `identity:get_project(project_id=28ee187de5744709bb65431943eb0200)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:48.190177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bd32e370-7c24-456d-b26d-3d42ad494de0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 14:56:48.190840 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32e370-7c24-456d-b26d-3d42ad494de0 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:48.195284 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 3/5] 10.222.0.93 () {56 vars in 1207 bytes} [Mon Aug 1 14:56:48 2022] GET /identity/v3/projects/28ee187de5744709bb65431943eb0200 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:56:50.218067 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f8c34-6ab2-4920-9bc1-536da88e6dde None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:50.218620 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f8c34-6ab2-4920-9bc1-536da88e6dde None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:50.218620 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f8c34-6ab2-4920-9bc1-536da88e6dde None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:50.222705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 3/6] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:56:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:56:50.228016 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06693ec-d460-4fe0-aab9-1396ffe33f61 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:50.228443 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06693ec-d460-4fe0-aab9-1396ffe33f61 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:50.228704 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06693ec-d460-4fe0-aab9-1396ffe33f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:50.248666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-b06693ec-d460-4fe0-aab9-1396ffe33f61 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:56:50.250201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b06693ec-d460-4fe0-aab9-1396ffe33f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:56:50.258042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 4/7] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:56:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 14:56:50.281125 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f8c34-6ab2-4920-9bc1-536da88e6dde None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:50.293937 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f8c34-6ab2-4920-9bc1-536da88e6dde None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:50.298185 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e34cb-8a56-43e5-8212-b836005be52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:50.298800 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e34cb-8a56-43e5-8212-b836005be52f admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:50.299082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e34cb-8a56-43e5-8212-b836005be52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:50.299368 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e34cb-8a56-43e5-8212-b836005be52f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:50.307335 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e34cb-8a56-43e5-8212-b836005be52f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:50.309140 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e34cb-8a56-43e5-8212-b836005be52f admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:50.318932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 4/8] 10.222.0.93 () {58 vars in 1324 bytes} [Mon Aug 1 14:56:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:56:50.325248 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06693ec-d460-4fe0-aab9-1396ffe33f61 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:50.335558 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06693ec-d460-4fe0-aab9-1396ffe33f61 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:50.340641 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bdf9d-dc07-47b9-8d62-7871a0e849c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:50.341268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bdf9d-dc07-47b9-8d62-7871a0e849c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:50.341546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bdf9d-dc07-47b9-8d62-7871a0e849c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:50.341815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bdf9d-dc07-47b9-8d62-7871a0e849c4 admin admin] PATH_INFO: `/v3/users/b035f76541b24134b8802d5327d7615c` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:50.343177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66bdf9d-dc07-47b9-8d62-7871a0e849c4 admin admin] RBAC: Authorizing `identity:get_user(user_id=b035f76541b24134b8802d5327d7615c)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:50.345035 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66bdf9d-dc07-47b9-8d62-7871a0e849c4 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:50.346634 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 5/9] 10.222.0.93 () {56 vars in 1198 bytes} [Mon Aug 1 14:56:50 2022] GET /identity/v3/users/b035f76541b24134b8802d5327d7615c => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:56:52.462068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f77a2e-4e58-4846-aff9-8e10d5e98984 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:52.462588 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f77a2e-4e58-4846-aff9-8e10d5e98984 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:52.462695 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f77a2e-4e58-4846-aff9-8e10d5e98984 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:52.463948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 5/10] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:56:52.469161 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33c308-f2d7-48b9-89b6-e686631d07f5 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:52.469602 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33c308-f2d7-48b9-89b6-e686631d07f5 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:52.469722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33c308-f2d7-48b9-89b6-e686631d07f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:52.484916 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b397e6-c5a1-4058-b2ef-9d7c0c254989 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:52.485278 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b397e6-c5a1-4058-b2ef-9d7c0c254989 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:52.485637 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b397e6-c5a1-4058-b2ef-9d7c0c254989 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:52.487114 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 6/11] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:56:52.489696 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-4a33c308-f2d7-48b9-89b6-e686631d07f5 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:56:52.491109 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-4a33c308-f2d7-48b9-89b6-e686631d07f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:56:52.492323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a20258-8f27-41fa-943c-fc1f27927758 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:52.492722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a20258-8f27-41fa-943c-fc1f27927758 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:52.493483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a20258-8f27-41fa-943c-fc1f27927758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:52.536871 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG passlib.handlers.bcrypt [None req-87a20258-8f27-41fa-943c-fc1f27927758 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80412) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 14:56:52.537312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG passlib.handlers.bcrypt [None req-87a20258-8f27-41fa-943c-fc1f27927758 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80412) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 14:56:52.576001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 6/12] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:56:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:56:52.584167 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-87a20258-8f27-41fa-943c-fc1f27927758 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:56:52.585827 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-87a20258-8f27-41fa-943c-fc1f27927758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:56:52.598588 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33c308-f2d7-48b9-89b6-e686631d07f5 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:52.624139 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:56:52.690686 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 7/13] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:56:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:56:52.705557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33c308-f2d7-48b9-89b6-e686631d07f5 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:52.712775 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a20258-8f27-41fa-943c-fc1f27927758 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:52.740092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e84e44-6008-4e69-8851-5a0781027eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:52.740738 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e84e44-6008-4e69-8851-5a0781027eea None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:52.741034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e84e44-6008-4e69-8851-5a0781027eea None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:52.741294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e84e44-6008-4e69-8851-5a0781027eea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:52.753325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e84e44-6008-4e69-8851-5a0781027eea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:56:52.762134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-46e84e44-6008-4e69-8851-5a0781027eea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:56:52.762134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:56:52.762134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:56:52.762134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:56:52.762134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:56:52.762134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:56:52.762134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:56:52.762134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:56:52.762134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:56:52.762134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:56:52.762134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:56:52.762134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:56:52.762134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:56:52.762134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:56:52.762134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:56:52.762134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:56:52.763280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:56:52.763280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:56:52.763280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:56:52.763280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:56:52.763280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:56:52.763280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:56:52.763280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:56:52.763280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:56:52.763280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:56:52.763280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:56:52.763280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:56:52.763280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:56:52.763280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:56:52.763280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:56:52.763280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:56:52.763280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:56:52.764167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:56:52.764167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:56:52.764167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:56:52.764167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:56:52.764167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:56:52.764167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:56:52.764167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:56:52.764167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:56:52.764167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:56:52.764167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:56:52.764167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:56:52.764167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:56:52.764167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 7/14] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:56:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:56:52.769317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e84e44-6008-4e69-8851-5a0781027eea None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:52.815508 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a20258-8f27-41fa-943c-fc1f27927758 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:52.817848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:56:52.858891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d049650-1df7-4def-9843-ac0a9e356889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:52.859786 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d049650-1df7-4def-9843-ac0a9e356889 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:52.860080 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d049650-1df7-4def-9843-ac0a9e356889 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:52.860355 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d049650-1df7-4def-9843-ac0a9e356889 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:52.862749 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:56:52.883225 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e84e44-6008-4e69-8851-5a0781027eea None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:52.918378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63728f1-05d5-4cfc-8dc2-d64e97440489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:52.919133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63728f1-05d5-4cfc-8dc2-d64e97440489 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:52.919348 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63728f1-05d5-4cfc-8dc2-d64e97440489 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:52.919540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63728f1-05d5-4cfc-8dc2-d64e97440489 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:52.920256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63728f1-05d5-4cfc-8dc2-d64e97440489 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:56:52.920583 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63728f1-05d5-4cfc-8dc2-d64e97440489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:52.921988 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63728f1-05d5-4cfc-8dc2-d64e97440489 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:52.933067 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d049650-1df7-4def-9843-ac0a9e356889 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:52.934602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d049650-1df7-4def-9843-ac0a9e356889 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:52.935574 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 8/15] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:56:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:56:52.940937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63728f1-05d5-4cfc-8dc2-d64e97440489 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:53.042276 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 8/16] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:56:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:56:53.048167 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d049650-1df7-4def-9843-ac0a9e356889 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:53.051419 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:56:53.149327 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63728f1-05d5-4cfc-8dc2-d64e97440489 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:53.156968 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d049650-1df7-4def-9843-ac0a9e356889 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:53.185956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3cf17-ac34-453d-9d06-f8044216e7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:53.186598 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3cf17-ac34-453d-9d06-f8044216e7ae None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:53.186872 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3cf17-ac34-453d-9d06-f8044216e7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:53.187142 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3cf17-ac34-453d-9d06-f8044216e7ae None admin] PATH_INFO: `/v3/users/b035f76541b24134b8802d5327d7615c` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:53.188313 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3cf17-ac34-453d-9d06-f8044216e7ae None admin] RBAC: Authorizing `identity:get_user(user_id=b035f76541b24134b8802d5327d7615c)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:53.189834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3cf17-ac34-453d-9d06-f8044216e7ae None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:53.191428 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 9/17] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:56:52 2022] GET /identity/v3/users/b035f76541b24134b8802d5327d7615c => generated 247 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:56:53.197270 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3cf17-ac34-453d-9d06-f8044216e7ae None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:53.198666 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8678d-e5b0-4e51-8bea-8dcd34a42bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:53.199540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8678d-e5b0-4e51-8bea-8dcd34a42bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:53.199820 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8678d-e5b0-4e51-8bea-8dcd34a42bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:53.200101 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8678d-e5b0-4e51-8bea-8dcd34a42bd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:53.212300 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:56:53.219771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8678d-e5b0-4e51-8bea-8dcd34a42bd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:53.221267 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8678d-e5b0-4e51-8bea-8dcd34a42bd8 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:53.240599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-77e8678d-e5b0-4e51-8bea-8dcd34a42bd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:56:53.240599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:56:53.240599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:56:53.240599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:56:53.240599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:56:53.240599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:56:53.240599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:56:53.240599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:56:53.240599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:56:53.240599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:56:53.240599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:56:53.240599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:56:53.240599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:56:53.240599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:56:53.240599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:56:53.241526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:56:53.241526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:56:53.241526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:56:53.241526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:56:53.241526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:56:53.241526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:56:53.241526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:56:53.241526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:56:53.241526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:56:53.241526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:56:53.241526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:56:53.241526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:56:53.241526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:56:53.241526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:56:53.241526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:56:53.241526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:56:53.241526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:56:53.242342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:56:53.242342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:56:53.242342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:56:53.242342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:56:53.242342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 9/18] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:56:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:56:53.248547 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8678d-e5b0-4e51-8bea-8dcd34a42bd8 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:53.302931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3cf17-ac34-453d-9d06-f8044216e7ae None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:53.306063 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:56:53.342701 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99170bc0-ab77-4fe3-a3fc-38a615288f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:53.343368 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99170bc0-ab77-4fe3-a3fc-38a615288f19 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:53.344115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99170bc0-ab77-4fe3-a3fc-38a615288f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:53.344397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99170bc0-ab77-4fe3-a3fc-38a615288f19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:53.345988 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99170bc0-ab77-4fe3-a3fc-38a615288f19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:53.347390 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99170bc0-ab77-4fe3-a3fc-38a615288f19 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:53.348719 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 10/19] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:56:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:56:53.354979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99170bc0-ab77-4fe3-a3fc-38a615288f19 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:53.356582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8678d-e5b0-4e51-8bea-8dcd34a42bd8 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:53.392066 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299bec0-fd1f-4d9d-a146-c3874adfe5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:53.392711 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299bec0-fd1f-4d9d-a146-c3874adfe5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:53.392988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299bec0-fd1f-4d9d-a146-c3874adfe5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:53.393261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299bec0-fd1f-4d9d-a146-c3874adfe5d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:53.393944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299bec0-fd1f-4d9d-a146-c3874adfe5d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:56:53.394243 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299bec0-fd1f-4d9d-a146-c3874adfe5d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:53.395652 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299bec0-fd1f-4d9d-a146-c3874adfe5d4 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:53.413299 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 10/20] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:56:53.441742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:56:53.463134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99170bc0-ab77-4fe3-a3fc-38a615288f19 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:53.505339 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470e9c9-eb3a-4255-bc27-3d2ce1319211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:53.506011 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470e9c9-eb3a-4255-bc27-3d2ce1319211 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:53.506255 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470e9c9-eb3a-4255-bc27-3d2ce1319211 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:53.506521 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470e9c9-eb3a-4255-bc27-3d2ce1319211 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:53.507839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470e9c9-eb3a-4255-bc27-3d2ce1319211 None admin] RBAC: Adding query filter params (role.id=5456887a306649bf80a9ea7bd4282e97, scope.domain.id=default, user.id=b035f76541b24134b8802d5327d7615c) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:56:53.507839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470e9c9-eb3a-4255-bc27-3d2ce1319211 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:53.509219 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470e9c9-eb3a-4255-bc27-3d2ce1319211 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:53.517142 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 11/21] 10.222.0.93 () {56 vars in 1322 bytes} [Mon Aug 1 14:56:53 2022] GET /identity/v3/role_assignments?user.id=b035f76541b24134b8802d5327d7615c&scope.domain.id=default&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 233 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:56:55.919876 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205116c4-8495-4771-82d2-ff2d0e7df9d9 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:55.920418 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205116c4-8495-4771-82d2-ff2d0e7df9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:55.920418 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205116c4-8495-4771-82d2-ff2d0e7df9d9 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:55.922247 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 11/22] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:56:55.928299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2ebc9-9c8a-49ca-9867-c15d6740ba1e None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:55.928880 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2ebc9-9c8a-49ca-9867-c15d6740ba1e None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:55.929090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2ebc9-9c8a-49ca-9867-c15d6740ba1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:55.956385 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-dbb2ebc9-9c8a-49ca-9867-c15d6740ba1e None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:56:55.958071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-dbb2ebc9-9c8a-49ca-9867-c15d6740ba1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:56:55.993856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8a768-5604-479a-add0-858c82659cd3 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:55.994488 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8a768-5604-479a-add0-858c82659cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:55.994960 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8a768-5604-479a-add0-858c82659cd3 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:55.996539 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 12/23] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:56:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:56:56.002299 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed3505-6f6e-447a-bce4-8e06282eb34b None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:56.002884 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed3505-6f6e-447a-bce4-8e06282eb34b None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:56.003376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed3505-6f6e-447a-bce4-8e06282eb34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:56.030397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-74ed3505-6f6e-447a-bce4-8e06282eb34b None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:56:56.033044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-74ed3505-6f6e-447a-bce4-8e06282eb34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:56:56.114281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 12/24] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:56:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:56:56.125646 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc152822-f5cf-4fec-bfc2-f2155657100a None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:56.126402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc152822-f5cf-4fec-bfc2-f2155657100a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:56.126884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc152822-f5cf-4fec-bfc2-f2155657100a None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:56.128623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 13/25] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:56:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:56:56.135217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089368a0-fdeb-4392-b861-2d57e226d3d1 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:56.135801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089368a0-fdeb-4392-b861-2d57e226d3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:56.136296 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089368a0-fdeb-4392-b861-2d57e226d3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:56.163244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-089368a0-fdeb-4392-b861-2d57e226d3d1 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:56:56.165566 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-089368a0-fdeb-4392-b861-2d57e226d3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:56:56.201239 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 13/26] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:56:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:56:56.202396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed3505-6f6e-447a-bce4-8e06282eb34b None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:56.296834 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:56:56.317743 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 14/27] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:56:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:56:56.318302 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089368a0-fdeb-4392-b861-2d57e226d3d1 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:56.332861 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:56:56.436772 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed3505-6f6e-447a-bce4-8e06282eb34b None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:56.483592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8c8a8-29c1-4f50-a212-455483dbf36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:56.484790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8c8a8-29c1-4f50-a212-455483dbf36a None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:56.485366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8c8a8-29c1-4f50-a212-455483dbf36a None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:56.486373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8c8a8-29c1-4f50-a212-455483dbf36a None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:56.487789 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8c8a8-29c1-4f50-a212-455483dbf36a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:56.489649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8c8a8-29c1-4f50-a212-455483dbf36a None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:56.601508 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089368a0-fdeb-4392-b861-2d57e226d3d1 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:56.683111 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39b6d3-3859-4654-af39-72a899995d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:56.683765 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39b6d3-3859-4654-af39-72a899995d2b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:56.684055 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39b6d3-3859-4654-af39-72a899995d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:56.684324 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39b6d3-3859-4654-af39-72a899995d2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:56.708771 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39b6d3-3859-4654-af39-72a899995d2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:56.710269 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39b6d3-3859-4654-af39-72a899995d2b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:56.738167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-7f39b6d3-3859-4654-af39-72a899995d2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:56:56.738167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:56:56.738167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:56:56.738167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:56:56.738167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:56:56.738167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:56:56.738167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:56:56.738167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:56:56.738167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:56:56.738167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:56:56.738167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:56:56.738167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:56:56.738167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:56:56.738167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:56:56.738167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:56:56.739064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:56:56.739064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:56:56.739064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:56:56.739064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:56:56.739064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:56:56.739064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:56:56.739064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:56:56.739064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:56:56.739064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:56:56.739064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:56:56.739064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:56:56.739064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:56:56.739064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:56:56.739064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:56:56.739064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:56:56.739064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:56:56.739064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:56:56.739883 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:56:56.739883 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:56:56.739883 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:56:56.739883 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:56:56.739883 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 15/28] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:56:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:56:56.740174 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39b6d3-3859-4654-af39-72a899995d2b None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:56.782507 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 14/29] 10.222.0.93 () {60 vars in 1131 bytes} [Mon Aug 1 14:56:56 2022] POST /identity/v3/roles => generated 219 bytes in 580 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:56:56.786569 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8c8a8-29c1-4f50-a212-455483dbf36a None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:56.889231 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:56:56.968929 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39b6d3-3859-4654-af39-72a899995d2b None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:57.013443 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749d31c-bdfd-499c-b323-be712577034a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:57.014038 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749d31c-bdfd-499c-b323-be712577034a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:57.014334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749d31c-bdfd-499c-b323-be712577034a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:57.014621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749d31c-bdfd-499c-b323-be712577034a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:57.028301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8c8a8-29c1-4f50-a212-455483dbf36a None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:57.031049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749d31c-bdfd-499c-b323-be712577034a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:56:57.037897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-6749d31c-bdfd-499c-b323-be712577034a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:56:57.037897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:56:57.037897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:56:57.037897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:56:57.037897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:56:57.037897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:56:57.037897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:56:57.037897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:56:57.037897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:56:57.037897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:56:57.037897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:56:57.037897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:56:57.037897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:56:57.037897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:56:57.037897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:56:57.037897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:56:57.038830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:56:57.038830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:56:57.038830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:56:57.038830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:56:57.038830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:56:57.038830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:56:57.038830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:56:57.038830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:56:57.038830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:56:57.038830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:56:57.038830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:56:57.038830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:56:57.038830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:56:57.038830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:56:57.038830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:56:57.038830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:56:57.039666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:56:57.039666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:56:57.039666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:56:57.039666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:56:57.039666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:56:57.039666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:56:57.039666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:56:57.039666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:56:57.039666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:56:57.039666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:56:57.039666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:56:57.039666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:56:57.039666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 16/30] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:56:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:56:57.045248 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944aad0-23cf-4517-a408-082927711aa0 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:57.045642 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944aad0-23cf-4517-a408-082927711aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:57.045935 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944aad0-23cf-4517-a408-082927711aa0 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:57.047430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 17/31] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:56:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:56:57.048232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944aad0-23cf-4517-a408-082927711aa0 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:57.052010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:56:57.098043 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89857a32-1f8d-4732-a533-da19ac0d4ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:57.098693 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89857a32-1f8d-4732-a533-da19ac0d4ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:57.098974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89857a32-1f8d-4732-a533-da19ac0d4ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:57.099251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89857a32-1f8d-4732-a533-da19ac0d4ba9 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:57.100066 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89857a32-1f8d-4732-a533-da19ac0d4ba9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:56:57.100374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89857a32-1f8d-4732-a533-da19ac0d4ba9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:57.102159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89857a32-1f8d-4732-a533-da19ac0d4ba9 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:57.117394 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 15/32] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:56:57.118991 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27034299-eb32-49df-b941-6214ababba97 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:57.119331 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27034299-eb32-49df-b941-6214ababba97 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:57.119594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27034299-eb32-49df-b941-6214ababba97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:57.146863 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-27034299-eb32-49df-b941-6214ababba97 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:56:57.150160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-27034299-eb32-49df-b941-6214ababba97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:56:57.184107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944aad0-23cf-4517-a408-082927711aa0 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:57.210520 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:56:57.240400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a4dfe-ef28-4b0d-96d2-0fe225b724df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:57.241053 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a4dfe-ef28-4b0d-96d2-0fe225b724df None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:57.241331 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a4dfe-ef28-4b0d-96d2-0fe225b724df None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:57.241604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a4dfe-ef28-4b0d-96d2-0fe225b724df None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:57.242136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 16/33] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:56:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:56:57.242324 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a4dfe-ef28-4b0d-96d2-0fe225b724df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:56:57.243183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a4dfe-ef28-4b0d-96d2-0fe225b724df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:57.243598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27034299-eb32-49df-b941-6214ababba97 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:57.244638 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a4dfe-ef28-4b0d-96d2-0fe225b724df None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:57.255836 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 18/34] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:56:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:56:57.262098 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a4dfe-ef28-4b0d-96d2-0fe225b724df None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:57.367025 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:56:57.376842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27034299-eb32-49df-b941-6214ababba97 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:57.381924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a4dfe-ef28-4b0d-96d2-0fe225b724df None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:57.417434 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935938f4-8fb1-4d17-b826-0b3c3a7dd435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:57.418079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935938f4-8fb1-4d17-b826-0b3c3a7dd435 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:57.418351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935938f4-8fb1-4d17-b826-0b3c3a7dd435 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:57.418620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935938f4-8fb1-4d17-b826-0b3c3a7dd435 None admin] PATH_INFO: `/v3/users/b035f76541b24134b8802d5327d7615c` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:57.420206 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935938f4-8fb1-4d17-b826-0b3c3a7dd435 None admin] RBAC: Authorizing `identity:get_user(user_id=b035f76541b24134b8802d5327d7615c)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:57.421298 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935938f4-8fb1-4d17-b826-0b3c3a7dd435 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:57.422288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be70cd-7f53-4857-9f14-9df4872116ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:57.422921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be70cd-7f53-4857-9f14-9df4872116ca None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:57.423044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 19/35] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:56:57 2022] GET /identity/v3/users/b035f76541b24134b8802d5327d7615c => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:56:57.423221 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be70cd-7f53-4857-9f14-9df4872116ca None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:57.423482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be70cd-7f53-4857-9f14-9df4872116ca None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:57.423648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935938f4-8fb1-4d17-b826-0b3c3a7dd435 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:57.424525 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be70cd-7f53-4857-9f14-9df4872116ca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:57.425986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be70cd-7f53-4857-9f14-9df4872116ca None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:57.457301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 17/36] 10.222.0.93 () {60 vars in 1141 bytes} [Mon Aug 1 14:56:57 2022] POST /identity/v3/projects => generated 301 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:56:57.458644 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a572ad1-d958-4bdf-b144-b75feb980cc4 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:57.458938 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a572ad1-d958-4bdf-b144-b75feb980cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:57.459219 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a572ad1-d958-4bdf-b144-b75feb980cc4 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:57.460501 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 18/37] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:56:57.464815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef89b3b-bb49-4739-a2fc-a7d0e12668b3 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:57.464815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef89b3b-bb49-4739-a2fc-a7d0e12668b3 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:57.464815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef89b3b-bb49-4739-a2fc-a7d0e12668b3 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:57.464815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 19/38] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:56:57.464815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef89b3b-bb49-4739-a2fc-a7d0e12668b3 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:57.487448 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:56:57.561404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935938f4-8fb1-4d17-b826-0b3c3a7dd435 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:57.595731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef89b3b-bb49-4739-a2fc-a7d0e12668b3 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:57.604312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176a547-5c2f-43b6-938c-f0327e3588cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:57.604988 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176a547-5c2f-43b6-938c-f0327e3588cc None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:57.605268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176a547-5c2f-43b6-938c-f0327e3588cc None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:57.605541 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176a547-5c2f-43b6-938c-f0327e3588cc None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:57.606294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176a547-5c2f-43b6-938c-f0327e3588cc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:57.607709 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176a547-5c2f-43b6-938c-f0327e3588cc None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:57.641375 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768c531-17c8-4c47-99b8-fef9b801f74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:57.642060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768c531-17c8-4c47-99b8-fef9b801f74c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:57.642347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768c531-17c8-4c47-99b8-fef9b801f74c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:57.642654 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768c531-17c8-4c47-99b8-fef9b801f74c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:57.644293 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768c531-17c8-4c47-99b8-fef9b801f74c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:57.645812 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768c531-17c8-4c47-99b8-fef9b801f74c None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:57.647497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 20/39] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:56:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:56:57.648756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd62d2-59ee-4211-ae7e-7f7381dcd2ee None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:57.649038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd62d2-59ee-4211-ae7e-7f7381dcd2ee None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:57.649314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd62d2-59ee-4211-ae7e-7f7381dcd2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:57.671840 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-32fd62d2-59ee-4211-ae7e-7f7381dcd2ee None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:56:57.673162 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-32fd62d2-59ee-4211-ae7e-7f7381dcd2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:56:57.687286 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:56:57.736208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 20/40] 10.222.0.93 () {60 vars in 1131 bytes} [Mon Aug 1 14:56:57 2022] POST /identity/v3/roles => generated 225 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:56:57.737700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eec93b-e1b8-45d6-83a9-c2118e9fef75 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:57.737995 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eec93b-e1b8-45d6-83a9-c2118e9fef75 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:57.738277 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eec93b-e1b8-45d6-83a9-c2118e9fef75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:57.755675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 21/41] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:56:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:56:57.757336 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd62d2-59ee-4211-ae7e-7f7381dcd2ee None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:57.759470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-72eec93b-e1b8-45d6-83a9-c2118e9fef75 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:56:57.761265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-72eec93b-e1b8-45d6-83a9-c2118e9fef75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:56:57.856271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 21/42] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:56:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:56:57.864693 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eec93b-e1b8-45d6-83a9-c2118e9fef75 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:57.903042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:56:57.908880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd62d2-59ee-4211-ae7e-7f7381dcd2ee None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:57.956082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f8713-a464-4267-b603-5027b05869ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:57.956836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f8713-a464-4267-b603-5027b05869ca None admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:57.957115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f8713-a464-4267-b603-5027b05869ca None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:57.957390 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f8713-a464-4267-b603-5027b05869ca None admin] PATH_INFO: `/v3/domains/default/users/b035f76541b24134b8802d5327d7615c/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:57.959709 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f8713-a464-4267-b603-5027b05869ca None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b035f76541b24134b8802d5327d7615c, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:57.961288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f8713-a464-4267-b603-5027b05869ca None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:57.976483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 22/43] 10.222.0.93 () {58 vars in 1361 bytes} [Mon Aug 1 14:56:57 2022] PUT /identity/v3/domains/default/users/b035f76541b24134b8802d5327d7615c/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:56:57.976698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eec93b-e1b8-45d6-83a9-c2118e9fef75 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:57.977866 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f8713-a464-4267-b603-5027b05869ca None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:57.999369 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:56:58.020288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a694e5a-9d91-467a-b4cb-6983c7965df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:58.021052 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a694e5a-9d91-467a-b4cb-6983c7965df6 None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:58.021318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a694e5a-9d91-467a-b4cb-6983c7965df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:58.021597 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a694e5a-9d91-467a-b4cb-6983c7965df6 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:58.022318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a694e5a-9d91-467a-b4cb-6983c7965df6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:58.024419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a694e5a-9d91-467a-b4cb-6983c7965df6 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:58.046060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 22/44] 10.222.0.93 () {60 vars in 1131 bytes} [Mon Aug 1 14:56:57 2022] POST /identity/v3/roles => generated 223 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:56:58.138668 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f8713-a464-4267-b603-5027b05869ca None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:58.184993 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba81366-1cf0-47b8-b9a2-a067ac5ca5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:58.185883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba81366-1cf0-47b8-b9a2-a067ac5ca5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:58.186315 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba81366-1cf0-47b8-b9a2-a067ac5ca5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:58.186730 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba81366-1cf0-47b8-b9a2-a067ac5ca5a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:58.188154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba81366-1cf0-47b8-b9a2-a067ac5ca5a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:58.189871 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba81366-1cf0-47b8-b9a2-a067ac5ca5a0 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:58.191703 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 23/45] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:56:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:56:58.197859 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a694e5a-9d91-467a-b4cb-6983c7965df6 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:56:58.302512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:56:58.309355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a694e5a-9d91-467a-b4cb-6983c7965df6 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:56:58.344684 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ba214-bd2b-4094-800d-85438c44bc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:56:58.344885 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ba214-bd2b-4094-800d-85438c44bc58 None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:56:58.345021 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ba214-bd2b-4094-800d-85438c44bc58 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:56:58.345272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ba214-bd2b-4094-800d-85438c44bc58 None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:56:58.345956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ba214-bd2b-4094-800d-85438c44bc58 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:56:58.347403 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ba214-bd2b-4094-800d-85438c44bc58 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:56:58.376217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 23/46] 10.222.0.93 () {60 vars in 1141 bytes} [Mon Aug 1 14:56:58 2022] POST /identity/v3/projects => generated 312 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:57:00.046994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af3786-4ac7-4675-8cf1-5066c39bbd50 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:00.047552 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af3786-4ac7-4675-8cf1-5066c39bbd50 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:00.047552 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af3786-4ac7-4675-8cf1-5066c39bbd50 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:00.049120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 24/47] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:00.054875 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58662d7a-4f25-4be1-b19a-1b914e3331b4 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:00.055391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58662d7a-4f25-4be1-b19a-1b914e3331b4 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:00.055498 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58662d7a-4f25-4be1-b19a-1b914e3331b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:00.083999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-58662d7a-4f25-4be1-b19a-1b914e3331b4 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:00.085275 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-58662d7a-4f25-4be1-b19a-1b914e3331b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:00.163731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 24/48] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:00.186492 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af3786-4ac7-4675-8cf1-5066c39bbd50 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:00.271457 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:00.305682 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af3786-4ac7-4675-8cf1-5066c39bbd50 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:00.344627 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c998a-500c-419e-af13-cfad22b668bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:00.345668 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c998a-500c-419e-af13-cfad22b668bb None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:00.345668 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c998a-500c-419e-af13-cfad22b668bb None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:00.345926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c998a-500c-419e-af13-cfad22b668bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:00.358631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c998a-500c-419e-af13-cfad22b668bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:00.364902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-c37c998a-500c-419e-af13-cfad22b668bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:57:00.364902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:00.364902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:00.364902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:00.364902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:00.364902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:00.364902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:00.364902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:00.364902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:00.364902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:00.364902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:00.364902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:00.364902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:00.364902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:00.364902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:00.364902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:00.366551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:00.366551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:00.366551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:00.366551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:00.366551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:00.366551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:00.366551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:00.366551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:00.366551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:00.366551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:00.366551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:00.366551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:00.366551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:00.366551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:00.366551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:00.366551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:00.367955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:00.367955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:00.367955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:00.367955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:00.367955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:00.367955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:00.367955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:00.367955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:00.367955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:00.367955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:00.367955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:57:00.367955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:00.367955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 25/49] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:57:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:00.376455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58662d7a-4f25-4be1-b19a-1b914e3331b4 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:00.395051 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55738608-856f-4393-a3d3-149f0788adcf None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:00.395465 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55738608-856f-4393-a3d3-149f0788adcf None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:00.395620 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55738608-856f-4393-a3d3-149f0788adcf None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:00.397032 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 26/50] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:57:00.403387 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3afb1-78fd-4879-a5d1-0a13e9b57ed4 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:00.403387 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3afb1-78fd-4879-a5d1-0a13e9b57ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:00.403647 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3afb1-78fd-4879-a5d1-0a13e9b57ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:00.424177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-83a3afb1-78fd-4879-a5d1-0a13e9b57ed4 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:00.425462 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-83a3afb1-78fd-4879-a5d1-0a13e9b57ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:00.470394 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:00.503591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 27/51] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:57:00.504489 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58662d7a-4f25-4be1-b19a-1b914e3331b4 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:00.526632 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3afb1-78fd-4879-a5d1-0a13e9b57ed4 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:00.546696 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8b989-e143-4a5b-bd36-7ac111f540c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:00.547637 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8b989-e143-4a5b-bd36-7ac111f540c9 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:00.547905 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8b989-e143-4a5b-bd36-7ac111f540c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:00.548183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8b989-e143-4a5b-bd36-7ac111f540c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:00.548945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8b989-e143-4a5b-bd36-7ac111f540c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:00.549259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8b989-e143-4a5b-bd36-7ac111f540c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:00.550666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8b989-e143-4a5b-bd36-7ac111f540c9 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:00.559851 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 25/52] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:00.566295 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8b989-e143-4a5b-bd36-7ac111f540c9 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:00.592868 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:00.640676 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3afb1-78fd-4879-a5d1-0a13e9b57ed4 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:00.684011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84092c-3a64-4cec-8213-7a107cb5dcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:00.684874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84092c-3a64-4cec-8213-7a107cb5dcef None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:00.685254 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84092c-3a64-4cec-8213-7a107cb5dcef None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:00.685543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84092c-3a64-4cec-8213-7a107cb5dcef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:00.687088 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84092c-3a64-4cec-8213-7a107cb5dcef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:00.688651 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84092c-3a64-4cec-8213-7a107cb5dcef None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:00.690246 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8b989-e143-4a5b-bd36-7ac111f540c9 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:00.690412 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 28/53] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:57:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:00.696572 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84092c-3a64-4cec-8213-7a107cb5dcef None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:00.732188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178500ce-e288-41a4-a986-b0797098185c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:00.732936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178500ce-e288-41a4-a986-b0797098185c None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:00.735555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178500ce-e288-41a4-a986-b0797098185c None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:00.735555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178500ce-e288-41a4-a986-b0797098185c None admin] PATH_INFO: `/v3/users/b035f76541b24134b8802d5327d7615c` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:00.735555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178500ce-e288-41a4-a986-b0797098185c None admin] RBAC: Authorizing `identity:get_user(user_id=b035f76541b24134b8802d5327d7615c)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:00.736224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178500ce-e288-41a4-a986-b0797098185c None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:00.737932 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 26/54] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:00 2022] GET /identity/v3/users/b035f76541b24134b8802d5327d7615c => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:00.745377 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178500ce-e288-41a4-a986-b0797098185c None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:00.763649 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:00.847001 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84092c-3a64-4cec-8213-7a107cb5dcef None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:00.886328 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178500ce-e288-41a4-a986-b0797098185c None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:00.887503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:00.896511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099d18d-b255-4170-8dd3-770f01561693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:00.897273 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099d18d-b255-4170-8dd3-770f01561693 None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:00.897557 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099d18d-b255-4170-8dd3-770f01561693 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:00.897942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099d18d-b255-4170-8dd3-770f01561693 None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:00.898557 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099d18d-b255-4170-8dd3-770f01561693 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:00.901166 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099d18d-b255-4170-8dd3-770f01561693 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:00.932926 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82a17e-96d8-41e8-b76d-dd458c0fa562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:00.933777 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82a17e-96d8-41e8-b76d-dd458c0fa562 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:00.934216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82a17e-96d8-41e8-b76d-dd458c0fa562 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:00.934663 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82a17e-96d8-41e8-b76d-dd458c0fa562 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:00.936417 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82a17e-96d8-41e8-b76d-dd458c0fa562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:00.937679 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 29/55] 10.222.0.93 () {60 vars in 1140 bytes} [Mon Aug 1 14:57:00 2022] POST /identity/v3/projects => generated 298 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:57:00.938503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82a17e-96d8-41e8-b76d-dd458c0fa562 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:00.940615 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 27/56] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:57:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:00.948553 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099d18d-b255-4170-8dd3-770f01561693 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:01.109696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099d18d-b255-4170-8dd3-770f01561693 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:01.152140 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c82f6-2d36-409c-9b37-5a22d84e585c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:01.152903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c82f6-2d36-409c-9b37-5a22d84e585c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:01.153218 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c82f6-2d36-409c-9b37-5a22d84e585c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:01.153506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c82f6-2d36-409c-9b37-5a22d84e585c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:01.154319 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c82f6-2d36-409c-9b37-5a22d84e585c None admin] RBAC: Adding query filter params (role.id=5456887a306649bf80a9ea7bd4282e97, scope.domain.id=default, user.id=b035f76541b24134b8802d5327d7615c) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:01.154627 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c82f6-2d36-409c-9b37-5a22d84e585c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:01.156278 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c82f6-2d36-409c-9b37-5a22d84e585c None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:01.166288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 30/57] 10.222.0.93 () {56 vars in 1322 bytes} [Mon Aug 1 14:57:00 2022] GET /identity/v3/role_assignments?user.id=b035f76541b24134b8802d5327d7615c&scope.domain.id=default&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 530 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:03.038041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f289b4-f32d-442a-910d-95ca8b62f26e None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:03.038570 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f289b4-f32d-442a-910d-95ca8b62f26e None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:03.038659 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f289b4-f32d-442a-910d-95ca8b62f26e None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:03.040687 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 28/58] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:03.046045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb81cb9-e870-4364-a6dd-c553dc2e50db None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:03.046513 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb81cb9-e870-4364-a6dd-c553dc2e50db None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:03.046602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb81cb9-e870-4364-a6dd-c553dc2e50db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:03.066465 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-fdb81cb9-e870-4364-a6dd-c553dc2e50db None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:03.067702 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-fdb81cb9-e870-4364-a6dd-c553dc2e50db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:03.115708 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:03.141178 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 31/59] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:03.166377 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f289b4-f32d-442a-910d-95ca8b62f26e None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:03.249710 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:03.274838 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f289b4-f32d-442a-910d-95ca8b62f26e None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:03.313127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2b958d-60f3-46c5-8a2e-22ef51822e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:03.313781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b958d-60f3-46c5-8a2e-22ef51822e76 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:03.314149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b958d-60f3-46c5-8a2e-22ef51822e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:03.314333 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b958d-60f3-46c5-8a2e-22ef51822e76 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:03.315638 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2b958d-60f3-46c5-8a2e-22ef51822e76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:03.317267 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2b958d-60f3-46c5-8a2e-22ef51822e76 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:03.318957 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 29/60] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:57:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:03.324921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb81cb9-e870-4364-a6dd-c553dc2e50db None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:03.438011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb81cb9-e870-4364-a6dd-c553dc2e50db None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:03.484403 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5eebf-f204-4e58-a10a-04426461c976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:03.485108 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5eebf-f204-4e58-a10a-04426461c976 None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:03.485392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5eebf-f204-4e58-a10a-04426461c976 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:03.485672 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5eebf-f204-4e58-a10a-04426461c976 None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:03.486401 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d5eebf-f204-4e58-a10a-04426461c976 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:03.488014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d5eebf-f204-4e58-a10a-04426461c976 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:03.519655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 32/61] 10.222.0.93 () {60 vars in 1132 bytes} [Mon Aug 1 14:57:03 2022] POST /identity/v3/users => generated 275 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:57:06.879157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d24ec2-3707-44ab-888c-c3a490d03ce0 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:06.880508 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d24ec2-3707-44ab-888c-c3a490d03ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:06.881003 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d24ec2-3707-44ab-888c-c3a490d03ce0 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:06.883391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 30/62] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:06.889842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecb50b-98bf-44b9-a5f4-3bc52328b951 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:06.890982 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecb50b-98bf-44b9-a5f4-3bc52328b951 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:06.891457 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecb50b-98bf-44b9-a5f4-3bc52328b951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:06.919929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-1eecb50b-98bf-44b9-a5f4-3bc52328b951 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:06.921847 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-1eecb50b-98bf-44b9-a5f4-3bc52328b951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:06.967099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:07.023230 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 33/63] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:07.056081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d24ec2-3707-44ab-888c-c3a490d03ce0 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:07.157678 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4c262-8872-4a87-bfe5-91f3e43080a5 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:07.157861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4c262-8872-4a87-bfe5-91f3e43080a5 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:07.158099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4c262-8872-4a87-bfe5-91f3e43080a5 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:07.159916 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 34/64] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:07.166013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cdd57-2a56-494d-a459-dfeeefb46d52 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:07.166359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cdd57-2a56-494d-a459-dfeeefb46d52 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:07.166593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cdd57-2a56-494d-a459-dfeeefb46d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:07.202616 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-807cdd57-2a56-494d-a459-dfeeefb46d52 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:07.203840 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-807cdd57-2a56-494d-a459-dfeeefb46d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:07.256235 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d24ec2-3707-44ab-888c-c3a490d03ce0 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:07.339493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a695020-129e-4c42-a27d-a09d292a5848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:07.340151 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a695020-129e-4c42-a27d-a09d292a5848 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:07.340439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a695020-129e-4c42-a27d-a09d292a5848 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:07.340750 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a695020-129e-4c42-a27d-a09d292a5848 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:07.363606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 35/65] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:57:07.366760 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a695020-129e-4c42-a27d-a09d292a5848 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:07.378512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-8a695020-129e-4c42-a27d-a09d292a5848 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:07.378512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:07.378512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:07.378512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:07.378512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:07.378512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:07.378512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:07.378512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:07.378512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:07.378512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:07.378512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:07.378512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:07.378512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:07.378512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:07.378512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:07.378512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:07.379759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:07.379759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:07.379759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:07.379759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:07.379759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:07.379759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:07.379759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:07.379759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:07.379759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:07.379759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:07.379759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:07.379759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:07.379759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:07.379759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:07.379759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:07.379759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:07.381057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:07.381057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:07.381057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:07.381057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:07.381057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:07.381057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:07.381057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:07.381057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:07.381057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:07.381057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:07.381057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:07.381057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:07.385228 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 31/66] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:07.392932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807cdd57-2a56-494d-a459-dfeeefb46d52 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:07.403169 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a695020-129e-4c42-a27d-a09d292a5848 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:07.417036 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:07.591855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:07.602936 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807cdd57-2a56-494d-a459-dfeeefb46d52 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:07.613915 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a695020-129e-4c42-a27d-a09d292a5848 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:07.652705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a5b48-5eef-44e4-8bd7-d976ecf50d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:07.653620 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a5b48-5eef-44e4-8bd7-d976ecf50d64 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:07.654049 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a5b48-5eef-44e4-8bd7-d976ecf50d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:07.654570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a5b48-5eef-44e4-8bd7-d976ecf50d64 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:07.655560 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a5b48-5eef-44e4-8bd7-d976ecf50d64 None admin] RBAC: Adding query filter params (name=member) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:07.656018 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a5b48-5eef-44e4-8bd7-d976ecf50d64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:07.657761 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a5b48-5eef-44e4-8bd7-d976ecf50d64 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:07.665060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c9c6c-46d3-42f5-8828-f1fda7f72b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:07.665791 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c9c6c-46d3-42f5-8828-f1fda7f72b9d None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:07.666081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c9c6c-46d3-42f5-8828-f1fda7f72b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:07.666380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c9c6c-46d3-42f5-8828-f1fda7f72b9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:07.674707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 36/67] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:07.676625 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d55b2-efd9-4c73-a34d-28b05641d291 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:07.677139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d55b2-efd9-4c73-a34d-28b05641d291 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:07.677678 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d55b2-efd9-4c73-a34d-28b05641d291 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:07.679390 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 37/68] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:07.684633 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d55b2-efd9-4c73-a34d-28b05641d291 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:07.687002 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c9c6c-46d3-42f5-8828-f1fda7f72b9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:07.702587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-7a1c9c6c-46d3-42f5-8828-f1fda7f72b9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:57:07.702587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:07.702587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:07.702587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:07.702587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:07.702587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:07.702587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:07.702587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:07.702587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:07.702587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:07.702587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:07.702587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:07.702587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:07.702587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:07.702587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:07.702587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:07.703690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:07.703690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:07.703690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:07.703690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:07.703690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:07.703690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:07.703690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:07.703690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:07.703690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:07.703690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:07.703690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:07.703690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:07.703690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:07.703690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:07.703690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:07.703690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:07.704612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:07.704612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:07.704612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:07.704612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:07.704612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:07.704612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:07.704612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:07.704612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:07.704612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:07.704612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:07.704612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:57:07.704612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:07.705224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 32/69] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:57:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:57:07.708240 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834f5cd-24ab-4689-848c-84a71e7b2c98 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:07.708399 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834f5cd-24ab-4689-848c-84a71e7b2c98 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:07.708731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834f5cd-24ab-4689-848c-84a71e7b2c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:07.738221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-4834f5cd-24ab-4689-848c-84a71e7b2c98 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:07.739766 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-4834f5cd-24ab-4689-848c-84a71e7b2c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:07.821913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:07.835288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 33/70] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:07.841793 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834f5cd-24ab-4689-848c-84a71e7b2c98 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:07.877643 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d55b2-efd9-4c73-a34d-28b05641d291 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:07.926883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffea45-5dd7-42fc-bb52-286430af1e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:07.927517 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffea45-5dd7-42fc-bb52-286430af1e28 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:07.927839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffea45-5dd7-42fc-bb52-286430af1e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:07.928083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffea45-5dd7-42fc-bb52-286430af1e28 None admin] PATH_INFO: `/v3/users/d87f64f8b42c4884ba624a882478570d` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:07.947699 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffea45-5dd7-42fc-bb52-286430af1e28 None admin] RBAC: Authorizing `identity:get_user(user_id=d87f64f8b42c4884ba624a882478570d)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:07.949611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffea45-5dd7-42fc-bb52-286430af1e28 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:07.951810 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 38/71] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:07 2022] GET /identity/v3/users/d87f64f8b42c4884ba624a882478570d => generated 275 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:07.952402 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffea45-5dd7-42fc-bb52-286430af1e28 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:07.971101 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:08.013282 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834f5cd-24ab-4689-848c-84a71e7b2c98 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:08.053213 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92860c55-666a-4413-8534-973acfd95fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:08.053881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92860c55-666a-4413-8534-973acfd95fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:08.054161 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92860c55-666a-4413-8534-973acfd95fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:08.054435 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92860c55-666a-4413-8534-973acfd95fb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:08.055678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92860c55-666a-4413-8534-973acfd95fb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:08.055978 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92860c55-666a-4413-8534-973acfd95fb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:08.057536 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92860c55-666a-4413-8534-973acfd95fb3 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:08.066942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 34/72] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:08.068261 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da2439-0d77-45dd-8870-ceefe2c197e9 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:08.068611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da2439-0d77-45dd-8870-ceefe2c197e9 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:08.068989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da2439-0d77-45dd-8870-ceefe2c197e9 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:08.073205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 35/73] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:08.073205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da2439-0d77-45dd-8870-ceefe2c197e9 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:08.090313 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffea45-5dd7-42fc-bb52-286430af1e28 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:08.139228 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897dc59-3e80-4814-8813-3f6d5ba03b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:08.139898 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897dc59-3e80-4814-8813-3f6d5ba03b1d None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:08.140179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897dc59-3e80-4814-8813-3f6d5ba03b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:08.140457 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897dc59-3e80-4814-8813-3f6d5ba03b1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:08.152273 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f897dc59-3e80-4814-8813-3f6d5ba03b1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:08.157568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-f897dc59-3e80-4814-8813-3f6d5ba03b1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:08.157568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:08.157568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:08.157568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:08.157568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:08.157568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:08.157568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:08.157568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:08.157568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:08.157568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:08.157568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:08.157568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:08.157568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:08.157568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:08.157568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:08.157568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:08.158579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:08.158579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:08.158579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:08.158579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:08.158579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:08.158579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:08.158579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:08.158579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:08.158579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:08.158579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:08.158579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:08.158579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:08.158579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:08.158579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:08.158579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:08.158579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:08.159531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:08.159531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:08.159531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:08.159531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:08.159531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:08.159531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:08.159531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:08.159531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:08.159531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:08.159531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:08.159531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:08.159531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:08.159531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 39/74] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:57:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:08.160854 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e16aa2-9d18-423f-bd5b-587c68279bf6 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:08.161510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e16aa2-9d18-423f-bd5b-587c68279bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:08.161510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e16aa2-9d18-423f-bd5b-587c68279bf6 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:08.162979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 40/75] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:08.163508 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e16aa2-9d18-423f-bd5b-587c68279bf6 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:08.171354 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:08.226322 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da2439-0d77-45dd-8870-ceefe2c197e9 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:08.258394 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:08.270334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5de7e2-fdbb-49c5-9bae-4194ec619151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:08.270979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5de7e2-fdbb-49c5-9bae-4194ec619151 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:08.271275 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5de7e2-fdbb-49c5-9bae-4194ec619151 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:08.271613 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5de7e2-fdbb-49c5-9bae-4194ec619151 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:08.292326 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e16aa2-9d18-423f-bd5b-587c68279bf6 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:08.341611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca37ffe-6bfb-4dc3-8f4b-35546fd7d6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:08.342292 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca37ffe-6bfb-4dc3-8f4b-35546fd7d6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:08.342573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca37ffe-6bfb-4dc3-8f4b-35546fd7d6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:08.343017 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca37ffe-6bfb-4dc3-8f4b-35546fd7d6e2 None admin] PATH_INFO: `/v3/users/b035f76541b24134b8802d5327d7615c` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:08.344265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca37ffe-6bfb-4dc3-8f4b-35546fd7d6e2 None admin] RBAC: Authorizing `identity:get_user(user_id=b035f76541b24134b8802d5327d7615c)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:08.345857 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca37ffe-6bfb-4dc3-8f4b-35546fd7d6e2 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:08.347640 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 41/76] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:08 2022] GET /identity/v3/users/b035f76541b24134b8802d5327d7615c => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:08.348888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8f4ce-1629-4421-b46a-f800767d4b40 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:08.349289 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8f4ce-1629-4421-b46a-f800767d4b40 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:08.349695 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8f4ce-1629-4421-b46a-f800767d4b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:08.355118 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5de7e2-fdbb-49c5-9bae-4194ec619151 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:08.361920 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5de7e2-fdbb-49c5-9bae-4194ec619151 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:08.371161 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-95b8f4ce-1629-4421-b46a-f800767d4b40 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:08.372688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-95b8f4ce-1629-4421-b46a-f800767d4b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:08.429192 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:08.470624 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 42/77] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:08.472341 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8f4ce-1629-4421-b46a-f800767d4b40 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:08.501602 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 36/78] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:57:08.503032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f3ab4-ce29-40c9-a4ab-921278ae5c67 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:08.503293 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f3ab4-ce29-40c9-a4ab-921278ae5c67 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:08.503590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f3ab4-ce29-40c9-a4ab-921278ae5c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:08.525444 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-7f9f3ab4-ce29-40c9-a4ab-921278ae5c67 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:08.526808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-7f9f3ab4-ce29-40c9-a4ab-921278ae5c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:08.588909 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:08.618560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 37/79] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:08.619511 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f3ab4-ce29-40c9-a4ab-921278ae5c67 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:08.635734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8f4ce-1629-4421-b46a-f800767d4b40 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:08.677342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92699d3-9b55-4f14-b659-1e129301cbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:08.677993 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92699d3-9b55-4f14-b659-1e129301cbf7 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:08.678255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92699d3-9b55-4f14-b659-1e129301cbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:08.678529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92699d3-9b55-4f14-b659-1e129301cbf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:08.679271 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92699d3-9b55-4f14-b659-1e129301cbf7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:08.679562 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92699d3-9b55-4f14-b659-1e129301cbf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:08.681038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92699d3-9b55-4f14-b659-1e129301cbf7 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:08.690196 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 43/80] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:57:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:08.691156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92699d3-9b55-4f14-b659-1e129301cbf7 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:08.731334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:08.754793 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f3ab4-ce29-40c9-a4ab-921278ae5c67 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:08.798625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ad21e-4807-4afd-93ea-342c2d0a2ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:08.799380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ad21e-4807-4afd-93ea-342c2d0a2ddd None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:08.799661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ad21e-4807-4afd-93ea-342c2d0a2ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:08.799943 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ad21e-4807-4afd-93ea-342c2d0a2ddd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:08.834619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92699d3-9b55-4f14-b659-1e129301cbf7 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:08.877332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ad21e-4807-4afd-93ea-342c2d0a2ddd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:08.878991 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ad21e-4807-4afd-93ea-342c2d0a2ddd None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:08.880988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b85ec-3eae-4fff-8c7a-253a0577e016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:08.881647 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b85ec-3eae-4fff-8c7a-253a0577e016 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:08.881919 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b85ec-3eae-4fff-8c7a-253a0577e016 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:08.882195 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b85ec-3eae-4fff-8c7a-253a0577e016 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:08.884010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b85ec-3eae-4fff-8c7a-253a0577e016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:08.885742 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b85ec-3eae-4fff-8c7a-253a0577e016 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:08.889877 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 44/81] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:57:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:08.889877 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b85ec-3eae-4fff-8c7a-253a0577e016 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:08.892583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:08.985305 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:09.007127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 38/82] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:57:09.007314 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ad21e-4807-4afd-93ea-342c2d0a2ddd None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:09.049456 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b85ec-3eae-4fff-8c7a-253a0577e016 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:09.103158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3801aa-6819-4703-89ae-04c651cea082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:09.103810 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3801aa-6819-4703-89ae-04c651cea082 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:09.104093 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3801aa-6819-4703-89ae-04c651cea082 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:09.104367 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3801aa-6819-4703-89ae-04c651cea082 None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:09.105652 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3801aa-6819-4703-89ae-04c651cea082 None admin] RBAC: Authorizing `identity:get_project(project_id=cc8867738cf24c5db418fa4bd4cd2c01)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:09.107248 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3801aa-6819-4703-89ae-04c651cea082 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:09.109107 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 45/83] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:08 2022] GET /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:09.111856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3801aa-6819-4703-89ae-04c651cea082 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:09.150377 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ad21e-4807-4afd-93ea-342c2d0a2ddd None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:09.204024 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6abe39-8345-4c2c-b855-7079c88efaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:09.204937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6abe39-8345-4c2c-b855-7079c88efaa0 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:09.205267 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6abe39-8345-4c2c-b855-7079c88efaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:09.205739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6abe39-8345-4c2c-b855-7079c88efaa0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:09.223514 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6abe39-8345-4c2c-b855-7079c88efaa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:09.226945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:09.231211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-0d6abe39-8345-4c2c-b855-7079c88efaa0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:09.231211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:09.231211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:09.231211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:09.231211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:09.231211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:09.231211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:09.231211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:09.231211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:09.231211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:09.231211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:09.231211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:09.231211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:09.231211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:09.231211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:09.231211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:09.233149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:09.233149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:09.233149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:09.233149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:09.233149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:09.233149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:09.233149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:09.233149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:09.233149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:09.233149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:09.233149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:09.233149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:09.233149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:09.233149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:09.233149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:09.233149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:09.234808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:09.234808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:09.234808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:09.234808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:09.234808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:09.234808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:09.234808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:09.234808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:09.234808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:09.234808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:09.234808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:09.234808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:09.234808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 39/84] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:09.234808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6abe39-8345-4c2c-b855-7079c88efaa0 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:09.271016 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3801aa-6819-4703-89ae-04c651cea082 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:09.322686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:09.327768 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b076bb-c6c9-4966-b365-f3ca980a47ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:09.328940 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b076bb-c6c9-4966-b365-f3ca980a47ab None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:09.329454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b076bb-c6c9-4966-b365-f3ca980a47ab None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:09.329878 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b076bb-c6c9-4966-b365-f3ca980a47ab None admin] PATH_INFO: `/v3/users/d87f64f8b42c4884ba624a882478570d` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:09.331821 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b076bb-c6c9-4966-b365-f3ca980a47ab None admin] RBAC: Authorizing `identity:get_user(user_id=d87f64f8b42c4884ba624a882478570d)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:09.333951 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b076bb-c6c9-4966-b365-f3ca980a47ab None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:09.336336 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 46/85] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:09 2022] GET /identity/v3/users/d87f64f8b42c4884ba624a882478570d => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:09.337729 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b076bb-c6c9-4966-b365-f3ca980a47ab None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:09.409706 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6abe39-8345-4c2c-b855-7079c88efaa0 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:09.466095 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b076bb-c6c9-4966-b365-f3ca980a47ab None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:09.470689 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8bb29b-d349-4123-947a-f08b6a97beb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:09.471390 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bb29b-d349-4123-947a-f08b6a97beb3 None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:09.471757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bb29b-d349-4123-947a-f08b6a97beb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:09.472223 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bb29b-d349-4123-947a-f08b6a97beb3 None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:09.473548 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8bb29b-d349-4123-947a-f08b6a97beb3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:09.476457 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8bb29b-d349-4123-947a-f08b6a97beb3 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:09.516277 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 40/86] 10.222.0.93 () {60 vars in 1132 bytes} [Mon Aug 1 14:57:09 2022] POST /identity/v3/users => generated 289 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:57:09.517189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8bb29b-d349-4123-947a-f08b6a97beb3 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:09.519007 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86e288-d590-4532-9219-4f0d534b75e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:09.520190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86e288-d590-4532-9219-4f0d534b75e6 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:09.520818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86e288-d590-4532-9219-4f0d534b75e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:09.521459 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86e288-d590-4532-9219-4f0d534b75e6 None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:09.522874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86e288-d590-4532-9219-4f0d534b75e6 None admin] RBAC: Authorizing `identity:get_project(project_id=cc8867738cf24c5db418fa4bd4cd2c01)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:09.524718 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86e288-d590-4532-9219-4f0d534b75e6 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:09.534781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 47/87] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:09 2022] GET /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:09.534781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86e288-d590-4532-9219-4f0d534b75e6 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:09.539699 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:09.672697 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86e288-d590-4532-9219-4f0d534b75e6 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:09.673477 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8bb29b-d349-4123-947a-f08b6a97beb3 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:09.675322 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:09.741040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379998c4-1a76-4555-a088-c53251b01818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:09.741916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379998c4-1a76-4555-a088-c53251b01818 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:09.743082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379998c4-1a76-4555-a088-c53251b01818 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:09.743559 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379998c4-1a76-4555-a088-c53251b01818 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:09.746065 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379998c4-1a76-4555-a088-c53251b01818 None admin] RBAC: Adding query filter params (role.id=c65b8ad9ade94760af715b3899bdd799, scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01, user.id=d87f64f8b42c4884ba624a882478570d) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:09.746574 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379998c4-1a76-4555-a088-c53251b01818 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:09.749338 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379998c4-1a76-4555-a088-c53251b01818 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:09.753324 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d062f7-bc51-47cd-9603-e0502046d678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:09.754003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d062f7-bc51-47cd-9603-e0502046d678 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:09.754348 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d062f7-bc51-47cd-9603-e0502046d678 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:09.754749 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d062f7-bc51-47cd-9603-e0502046d678 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:09.756759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d062f7-bc51-47cd-9603-e0502046d678 None admin] RBAC: Adding query filter params (name=member) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:09.756759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d062f7-bc51-47cd-9603-e0502046d678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:09.758606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d062f7-bc51-47cd-9603-e0502046d678 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:09.761076 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 41/88] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:57:09 2022] GET /identity/v3/role_assignments?user.id=d87f64f8b42c4884ba624a882478570d&scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01&role.id=c65b8ad9ade94760af715b3899bdd799 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:09.762818 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379998c4-1a76-4555-a088-c53251b01818 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:09.773200 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 48/89] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:09.774674 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d062f7-bc51-47cd-9603-e0502046d678 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:09.907559 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:09.970351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379998c4-1a76-4555-a088-c53251b01818 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:09.980054 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d062f7-bc51-47cd-9603-e0502046d678 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:10.019702 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03688123-b9cd-480c-9deb-2665dc99b571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:10.020414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03688123-b9cd-480c-9deb-2665dc99b571 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:10.020901 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03688123-b9cd-480c-9deb-2665dc99b571 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:10.021474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03688123-b9cd-480c-9deb-2665dc99b571 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:10.054876 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd9af1-0e5b-431f-80c8-c5165b93f4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:10.056016 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd9af1-0e5b-431f-80c8-c5165b93f4de None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:10.056696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd9af1-0e5b-431f-80c8-c5165b93f4de None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:10.057224 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd9af1-0e5b-431f-80c8-c5165b93f4de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:10.059002 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd9af1-0e5b-431f-80c8-c5165b93f4de None admin] RBAC: Adding query filter params (role.id=5456887a306649bf80a9ea7bd4282e97, scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01, user.id=b035f76541b24134b8802d5327d7615c) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:10.059713 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd9af1-0e5b-431f-80c8-c5165b93f4de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:10.062891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd9af1-0e5b-431f-80c8-c5165b93f4de None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:10.091973 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 49/90] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:57:09 2022] GET /identity/v3/role_assignments?user.id=b035f76541b24134b8802d5327d7615c&scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 259 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:10.093355 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd9af1-0e5b-431f-80c8-c5165b93f4de None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:10.131321 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:10.327864 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03688123-b9cd-480c-9deb-2665dc99b571 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:10.330821 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03688123-b9cd-480c-9deb-2665dc99b571 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:10.387329 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd9af1-0e5b-431f-80c8-c5165b93f4de None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:10.454733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:10.468629 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99c750-78d3-4d4a-9309-04165ff7b585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:10.470533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99c750-78d3-4d4a-9309-04165ff7b585 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:10.471321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99c750-78d3-4d4a-9309-04165ff7b585 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:10.472058 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99c750-78d3-4d4a-9309-04165ff7b585 None admin] PATH_INFO: `/v3/users/d87f64f8b42c4884ba624a882478570d` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:10.475119 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99c750-78d3-4d4a-9309-04165ff7b585 None admin] RBAC: Authorizing `identity:get_user(user_id=d87f64f8b42c4884ba624a882478570d)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:10.479683 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99c750-78d3-4d4a-9309-04165ff7b585 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:10.483294 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 50/91] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:10 2022] GET /identity/v3/users/d87f64f8b42c4884ba624a882478570d => generated 275 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:10.498384 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99c750-78d3-4d4a-9309-04165ff7b585 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:10.672724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 42/92] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 911 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:57:10.683779 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03688123-b9cd-480c-9deb-2665dc99b571 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:10.793633 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99c750-78d3-4d4a-9309-04165ff7b585 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:10.853455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:10.871757 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1639d-8cd5-4395-ab6d-d04502c7f25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:10.872691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1639d-8cd5-4395-ab6d-d04502c7f25e None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:10.872976 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1639d-8cd5-4395-ab6d-d04502c7f25e None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:10.873253 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1639d-8cd5-4395-ab6d-d04502c7f25e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:10.945663 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03688123-b9cd-480c-9deb-2665dc99b571 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:11.030464 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e1639d-8cd5-4395-ab6d-d04502c7f25e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:11.031953 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e1639d-8cd5-4395-ab6d-d04502c7f25e None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:11.034853 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:11.035217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5e90b-c80e-41f4-b34e-f9bf8fa84073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:11.036278 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5e90b-c80e-41f4-b34e-f9bf8fa84073 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:11.036399 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5e90b-c80e-41f4-b34e-f9bf8fa84073 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:11.036620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5e90b-c80e-41f4-b34e-f9bf8fa84073 None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:11.038960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5e90b-c80e-41f4-b34e-f9bf8fa84073 None admin] RBAC: Authorizing `identity:get_project(project_id=cc8867738cf24c5db418fa4bd4cd2c01)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:11.041044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5e90b-c80e-41f4-b34e-f9bf8fa84073 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:11.042988 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 43/93] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:10 2022] GET /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01 => generated 298 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:11.053079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5e90b-c80e-41f4-b34e-f9bf8fa84073 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:11.261555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5e90b-c80e-41f4-b34e-f9bf8fa84073 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:11.287562 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 51/94] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 790 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:57:11.301569 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1639d-8cd5-4395-ab6d-d04502c7f25e None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:11.316619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45662dfc-e9b7-44e1-927f-9518d0bb5d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:11.317641 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45662dfc-e9b7-44e1-927f-9518d0bb5d7c None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:11.318132 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45662dfc-e9b7-44e1-927f-9518d0bb5d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:11.318616 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45662dfc-e9b7-44e1-927f-9518d0bb5d7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:11.319632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45662dfc-e9b7-44e1-927f-9518d0bb5d7c None admin] RBAC: Adding query filter params (role.id=e880d7a2b8c74bb1beb5015723a4e3d7, scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01, user.id=d87f64f8b42c4884ba624a882478570d) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:11.320731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45662dfc-e9b7-44e1-927f-9518d0bb5d7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:11.323315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45662dfc-e9b7-44e1-927f-9518d0bb5d7c None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:11.326242 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:11.340574 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 44/95] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:57:11 2022] GET /identity/v3/role_assignments?user.id=d87f64f8b42c4884ba624a882478570d&scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01&role.id=e880d7a2b8c74bb1beb5015723a4e3d7 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:11.499599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1639d-8cd5-4395-ab6d-d04502c7f25e None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:11.501563 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:11.562315 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1ded0-3fc6-4143-80f0-ca6e855ed291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:11.562897 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1ded0-3fc6-4143-80f0-ca6e855ed291 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:11.563180 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1ded0-3fc6-4143-80f0-ca6e855ed291 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:11.563502 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1ded0-3fc6-4143-80f0-ca6e855ed291 None admin] PATH_INFO: `/v3/projects/ba14d307309e47ddb630ddc0ae0af22b` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:11.569525 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1ded0-3fc6-4143-80f0-ca6e855ed291 None admin] RBAC: Authorizing `identity:get_project(project_id=ba14d307309e47ddb630ddc0ae0af22b)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:11.571597 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1ded0-3fc6-4143-80f0-ca6e855ed291 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:11.573462 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 52/96] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:11 2022] GET /identity/v3/projects/ba14d307309e47ddb630ddc0ae0af22b => generated 312 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:11.583088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45662dfc-e9b7-44e1-927f-9518d0bb5d7c None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:11.773956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45662dfc-e9b7-44e1-927f-9518d0bb5d7c None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:11.830096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd617c8-f2fd-4d99-b31e-36d9dcbcc540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:11.830746 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd617c8-f2fd-4d99-b31e-36d9dcbcc540 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:11.831068 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd617c8-f2fd-4d99-b31e-36d9dcbcc540 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:11.831391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd617c8-f2fd-4d99-b31e-36d9dcbcc540 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:11.832361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd617c8-f2fd-4d99-b31e-36d9dcbcc540 None admin] RBAC: Adding query filter params (role.id=c65b8ad9ade94760af715b3899bdd799, scope.project.id=ba14d307309e47ddb630ddc0ae0af22b, user.id=d87f64f8b42c4884ba624a882478570d) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:11.832703 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd617c8-f2fd-4d99-b31e-36d9dcbcc540 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:11.834344 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd617c8-f2fd-4d99-b31e-36d9dcbcc540 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:11.844518 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 45/97] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:57:11 2022] GET /identity/v3/role_assignments?user.id=d87f64f8b42c4884ba624a882478570d&scope.project.id=ba14d307309e47ddb630ddc0ae0af22b&role.id=c65b8ad9ade94760af715b3899bdd799 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:13.378229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be453bab-594c-4337-9bdf-e220812f3248 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:13.379073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be453bab-594c-4337-9bdf-e220812f3248 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:13.379073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be453bab-594c-4337-9bdf-e220812f3248 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:13.380861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 53/98] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:13.391250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49447dc2-6c5c-4828-88da-115244225a62 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:13.392184 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49447dc2-6c5c-4828-88da-115244225a62 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:13.392544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49447dc2-6c5c-4828-88da-115244225a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:13.437492 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-49447dc2-6c5c-4828-88da-115244225a62 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:13.439727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-49447dc2-6c5c-4828-88da-115244225a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:13.565782 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:13.584768 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 46/99] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:13.586273 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42857905-6f59-4442-9934-f1bee1f442a5 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:13.586753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42857905-6f59-4442-9934-f1bee1f442a5 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:13.587220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42857905-6f59-4442-9934-f1bee1f442a5 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:13.589962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 54/100] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:13.596050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6560012-c3d5-4136-9f3b-ebcf87aa3808 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:13.596446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6560012-c3d5-4136-9f3b-ebcf87aa3808 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:13.596779 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6560012-c3d5-4136-9f3b-ebcf87aa3808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:13.624384 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-e6560012-c3d5-4136-9f3b-ebcf87aa3808 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:13.625799 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e6560012-c3d5-4136-9f3b-ebcf87aa3808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:13.674413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42857905-6f59-4442-9934-f1bee1f442a5 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:13.722503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 47/101] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:13.759389 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6560012-c3d5-4136-9f3b-ebcf87aa3808 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:14.062145 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6560012-c3d5-4136-9f3b-ebcf87aa3808 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:14.095014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42857905-6f59-4442-9934-f1bee1f442a5 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:14.116708 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:14.153605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d977af0-5cd1-43f3-87ef-93e2960ebeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:14.154669 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d977af0-5cd1-43f3-87ef-93e2960ebeab None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:14.155153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d977af0-5cd1-43f3-87ef-93e2960ebeab None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:14.155700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d977af0-5cd1-43f3-87ef-93e2960ebeab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:14.163531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d977af0-5cd1-43f3-87ef-93e2960ebeab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:14.167034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d977af0-5cd1-43f3-87ef-93e2960ebeab None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:14.170699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 48/102] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:57:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:14.172118 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095525e-9a23-423e-9d4b-ddf581405971 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:14.172418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095525e-9a23-423e-9d4b-ddf581405971 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:14.172743 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095525e-9a23-423e-9d4b-ddf581405971 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:14.174052 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 49/103] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:14.175127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35537b6d-c40f-4038-ab1f-2fbbe72f65fe None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:14.175419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35537b6d-c40f-4038-ab1f-2fbbe72f65fe None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:14.176051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35537b6d-c40f-4038-ab1f-2fbbe72f65fe None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:14.177340 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 50/104] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:14.177954 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35537b6d-c40f-4038-ab1f-2fbbe72f65fe None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:14.205974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6bc08-83de-4a64-986d-e717bfcce7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:14.206944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6bc08-83de-4a64-986d-e717bfcce7ac None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:14.207301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6bc08-83de-4a64-986d-e717bfcce7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:14.207606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6bc08-83de-4a64-986d-e717bfcce7ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:14.224550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6bc08-83de-4a64-986d-e717bfcce7ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:14.231767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-03b6bc08-83de-4a64-986d-e717bfcce7ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:57:14.231767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:14.231767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:14.231767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:14.231767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:14.231767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:14.231767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:14.231767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:14.231767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:14.231767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:14.231767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:14.231767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:14.231767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:14.231767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:14.231767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:14.231767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:14.234476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:14.234476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:14.234476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:14.234476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:14.234476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:14.234476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:14.234476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:14.234476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:14.234476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:14.234476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:14.234476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:14.234476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:14.234476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:14.234476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:14.234476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:14.234476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:14.236628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:14.236628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:14.236628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:14.236628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:14.236628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:14.236628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:14.236628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:14.236628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:14.236628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:14.236628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:14.236628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:57:14.236628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:14.236628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 55/105] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 560 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:14.244492 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7bab7-e7b8-47dd-9750-d5f459e86853 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:14.244492 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7bab7-e7b8-47dd-9750-d5f459e86853 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:14.244492 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7bab7-e7b8-47dd-9750-d5f459e86853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:14.269432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-b9e7bab7-e7b8-47dd-9750-d5f459e86853 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:14.271037 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b9e7bab7-e7b8-47dd-9750-d5f459e86853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:14.322580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:14.360064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35537b6d-c40f-4038-ab1f-2fbbe72f65fe None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:14.415064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 56/106] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:14.416729 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15593df0-6281-4f48-9f3d-b11498cfefb7 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:14.417025 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15593df0-6281-4f48-9f3d-b11498cfefb7 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:14.417314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15593df0-6281-4f48-9f3d-b11498cfefb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:14.443376 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98e813-d884-4e40-9fbd-b510da3847a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:14.444111 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98e813-d884-4e40-9fbd-b510da3847a4 None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:14.445146 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98e813-d884-4e40-9fbd-b510da3847a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:14.445910 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98e813-d884-4e40-9fbd-b510da3847a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:14.447883 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98e813-d884-4e40-9fbd-b510da3847a4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:14.449548 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98e813-d884-4e40-9fbd-b510da3847a4 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:14.457895 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-15593df0-6281-4f48-9f3d-b11498cfefb7 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:14.459732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-15593df0-6281-4f48-9f3d-b11498cfefb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:14.484819 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 51/107] 10.222.0.93 () {60 vars in 1141 bytes} [Mon Aug 1 14:57:14 2022] POST /identity/v3/projects => generated 302 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:57:14.486069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98e813-d884-4e40-9fbd-b510da3847a4 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:14.550128 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:14.632871 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 57/108] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:14.633758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15593df0-6281-4f48-9f3d-b11498cfefb7 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:14.673971 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98e813-d884-4e40-9fbd-b510da3847a4 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:14.760026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c1520-dedf-4cc3-b615-f574a3cf4417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:14.760747 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c1520-dedf-4cc3-b615-f574a3cf4417 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:14.761035 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c1520-dedf-4cc3-b615-f574a3cf4417 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:14.761317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c1520-dedf-4cc3-b615-f574a3cf4417 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:14.762171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c1520-dedf-4cc3-b615-f574a3cf4417 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:14.762589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c1520-dedf-4cc3-b615-f574a3cf4417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:14.764036 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c1520-dedf-4cc3-b615-f574a3cf4417 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:14.775549 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 52/109] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:14.776641 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c1520-dedf-4cc3-b615-f574a3cf4417 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:14.838387 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:14.857751 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15593df0-6281-4f48-9f3d-b11498cfefb7 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:14.914932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547ee60-d782-4ddf-9275-546ec7e89966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:14.917736 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547ee60-d782-4ddf-9275-546ec7e89966 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:14.918303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547ee60-d782-4ddf-9275-546ec7e89966 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:14.918942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547ee60-d782-4ddf-9275-546ec7e89966 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:14.939891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547ee60-d782-4ddf-9275-546ec7e89966 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:14.951003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-0547ee60-d782-4ddf-9275-546ec7e89966 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:14.951003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:14.951003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:14.951003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:14.951003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:14.951003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:14.951003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:14.951003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:14.951003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:14.951003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:14.951003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:14.951003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:14.951003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:14.951003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:14.951003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:14.951003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:14.952397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:14.952397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:14.952397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:14.952397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:14.952397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:14.952397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:14.952397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:14.952397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:14.952397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:14.952397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:14.952397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:14.952397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:14.952397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:14.952397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:14.952397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:14.952397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:14.953900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:14.953900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:14.953900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:14.953900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:14.953900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:14.953900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:14.953900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:14.953900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:14.953900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:14.953900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:14.953900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:14.953900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:14.957099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 58/110] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:14.958657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547ee60-d782-4ddf-9275-546ec7e89966 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:14.962833 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c1520-dedf-4cc3-b615-f574a3cf4417 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:14.965996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:15.011030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e52b8-317d-4c1e-b0e2-031315c02b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:15.012141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e52b8-317d-4c1e-b0e2-031315c02b76 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:15.012447 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e52b8-317d-4c1e-b0e2-031315c02b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:15.012828 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e52b8-317d-4c1e-b0e2-031315c02b76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:15.032504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e52b8-317d-4c1e-b0e2-031315c02b76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:15.032504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-0a4e52b8-317d-4c1e-b0e2-031315c02b76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:57:15.032504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:15.032504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:15.032504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:15.032504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:15.032504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:15.032504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:15.032504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:15.032504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:15.032504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:15.032504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:15.032504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:15.032504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:15.033688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:15.033688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:15.033688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:15.033688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:15.033688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:15.033688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:15.033688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:15.033688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:15.033688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:15.033688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:15.033688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:15.033688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:15.033688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:15.033688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:15.033688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:15.033688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:15.033688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:15.035178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:15.035178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:15.035178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:15.035178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:15.035178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:15.035178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:15.035178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:15.035178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:15.035178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:15.035178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:15.035178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:15.035178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:15.035178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:57:15.035178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:15.035178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 53/111] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:57:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:15.037573 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e52b8-317d-4c1e-b0e2-031315c02b76 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:15.176840 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547ee60-d782-4ddf-9275-546ec7e89966 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:15.218581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db083b06-aac7-4ea6-a1c2-c06a994af501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:15.219240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db083b06-aac7-4ea6-a1c2-c06a994af501 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:15.219530 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db083b06-aac7-4ea6-a1c2-c06a994af501 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:15.219849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db083b06-aac7-4ea6-a1c2-c06a994af501 None admin] PATH_INFO: `/v3/users/7699f51aaaca4af68243d0c6c54fc93d` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:15.235629 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db083b06-aac7-4ea6-a1c2-c06a994af501 None admin] RBAC: Authorizing `identity:get_user(user_id=7699f51aaaca4af68243d0c6c54fc93d)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:15.237396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db083b06-aac7-4ea6-a1c2-c06a994af501 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:15.239431 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 59/112] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:14 2022] GET /identity/v3/users/7699f51aaaca4af68243d0c6c54fc93d => generated 289 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:15.240120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db083b06-aac7-4ea6-a1c2-c06a994af501 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:15.289015 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:15.301471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e52b8-317d-4c1e-b0e2-031315c02b76 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:15.345890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13afebbd-e537-410c-b6a1-5a24d4f3c7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:15.346654 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afebbd-e537-410c-b6a1-5a24d4f3c7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:15.346988 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afebbd-e537-410c-b6a1-5a24d4f3c7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:15.347329 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afebbd-e537-410c-b6a1-5a24d4f3c7d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:15.348794 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13afebbd-e537-410c-b6a1-5a24d4f3c7d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:15.349165 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13afebbd-e537-410c-b6a1-5a24d4f3c7d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:15.351317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13afebbd-e537-410c-b6a1-5a24d4f3c7d6 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:15.368434 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 54/113] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:15.368434 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13afebbd-e537-410c-b6a1-5a24d4f3c7d6 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:15.391214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db083b06-aac7-4ea6-a1c2-c06a994af501 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:15.406747 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:15.451590 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1704ce6-5780-4037-8196-ab7febdd6a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:15.452407 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1704ce6-5780-4037-8196-ab7febdd6a2e None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:15.452583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1704ce6-5780-4037-8196-ab7febdd6a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:15.452808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1704ce6-5780-4037-8196-ab7febdd6a2e None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:15.453686 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1704ce6-5780-4037-8196-ab7febdd6a2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:15.454020 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1704ce6-5780-4037-8196-ab7febdd6a2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:15.455532 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1704ce6-5780-4037-8196-ab7febdd6a2e None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:15.467732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 60/114] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:15.469183 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f8d5f-bb2f-4c69-abca-2fb981f95b40 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:15.469466 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f8d5f-bb2f-4c69-abca-2fb981f95b40 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:15.469813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f8d5f-bb2f-4c69-abca-2fb981f95b40 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:15.471171 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 61/115] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:15.476596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e683d3e-e35e-41b4-8da8-0e11c06ccaa8 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:15.476596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e683d3e-e35e-41b4-8da8-0e11c06ccaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:15.476596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e683d3e-e35e-41b4-8da8-0e11c06ccaa8 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:15.476596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 62/116] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:15.481627 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e683d3e-e35e-41b4-8da8-0e11c06ccaa8 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:15.536632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13afebbd-e537-410c-b6a1-5a24d4f3c7d6 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:15.577854 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:15.594171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f2e73-aa52-458b-8433-26edcd2bc469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:15.594901 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f2e73-aa52-458b-8433-26edcd2bc469 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:15.595586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f2e73-aa52-458b-8433-26edcd2bc469 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:15.596152 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f2e73-aa52-458b-8433-26edcd2bc469 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:15.633997 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e683d3e-e35e-41b4-8da8-0e11c06ccaa8 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:15.689707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0c1c1-cf45-4c6a-8b0f-b12651e81934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:15.690496 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0c1c1-cf45-4c6a-8b0f-b12651e81934 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:15.690698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0c1c1-cf45-4c6a-8b0f-b12651e81934 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:15.690958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0c1c1-cf45-4c6a-8b0f-b12651e81934 None admin] PATH_INFO: `/v3/users/d87f64f8b42c4884ba624a882478570d` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:15.692240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0c1c1-cf45-4c6a-8b0f-b12651e81934 None admin] RBAC: Authorizing `identity:get_user(user_id=d87f64f8b42c4884ba624a882478570d)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:15.693776 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0c1c1-cf45-4c6a-8b0f-b12651e81934 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:15.696204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 63/117] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:15 2022] GET /identity/v3/users/d87f64f8b42c4884ba624a882478570d => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:15.698513 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58708172-3224-4070-a030-11c983bb35e6 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:15.699013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58708172-3224-4070-a030-11c983bb35e6 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:15.699556 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58708172-3224-4070-a030-11c983bb35e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:15.706355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f2e73-aa52-458b-8433-26edcd2bc469 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:15.708625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f2e73-aa52-458b-8433-26edcd2bc469 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:15.727137 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-58708172-3224-4070-a030-11c983bb35e6 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:15.728743 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-58708172-3224-4070-a030-11c983bb35e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:15.752733 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:15.850040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 64/118] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:15.853479 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90762e79-5928-4ccb-bee5-627650dbde75 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:15.853971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90762e79-5928-4ccb-bee5-627650dbde75 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:15.854448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90762e79-5928-4ccb-bee5-627650dbde75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:15.885074 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 55/119] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:57:15.886685 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f2e73-aa52-458b-8433-26edcd2bc469 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:15.891386 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-90762e79-5928-4ccb-bee5-627650dbde75 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:15.893622 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-90762e79-5928-4ccb-bee5-627650dbde75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:16.003474 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:16.017931 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 65/120] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:16.019514 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90762e79-5928-4ccb-bee5-627650dbde75 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:16.061115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f2e73-aa52-458b-8433-26edcd2bc469 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:16.102981 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:16.118569 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c45f6a-4128-4ef9-a8d3-782ea49bbf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:16.119531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c45f6a-4128-4ef9-a8d3-782ea49bbf3b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:16.119819 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c45f6a-4128-4ef9-a8d3-782ea49bbf3b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:16.120096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c45f6a-4128-4ef9-a8d3-782ea49bbf3b None admin] PATH_INFO: `/v3/users/b035f76541b24134b8802d5327d7615c` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:16.122696 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c45f6a-4128-4ef9-a8d3-782ea49bbf3b None admin] RBAC: Authorizing `identity:get_user(user_id=b035f76541b24134b8802d5327d7615c)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:16.124269 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c45f6a-4128-4ef9-a8d3-782ea49bbf3b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:16.126317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 56/121] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:15 2022] GET /identity/v3/users/b035f76541b24134b8802d5327d7615c => generated 247 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:16.127015 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c45f6a-4128-4ef9-a8d3-782ea49bbf3b None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:16.177134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90762e79-5928-4ccb-bee5-627650dbde75 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:16.234877 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60561424-cb23-4244-8508-ff98b063c22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:16.235579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60561424-cb23-4244-8508-ff98b063c22b None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:16.235866 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60561424-cb23-4244-8508-ff98b063c22b None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:16.236148 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60561424-cb23-4244-8508-ff98b063c22b None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:16.237523 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60561424-cb23-4244-8508-ff98b063c22b None admin] RBAC: Authorizing `identity:get_project(project_id=cc8867738cf24c5db418fa4bd4cd2c01)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:16.239064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60561424-cb23-4244-8508-ff98b063c22b None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:16.241014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 66/122] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:16 2022] GET /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:16.241874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60561424-cb23-4244-8508-ff98b063c22b None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:16.278968 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c45f6a-4128-4ef9-a8d3-782ea49bbf3b None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:16.319887 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b436b6-ad58-46b9-97b9-90200f0d1428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:16.320564 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:16.321049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b436b6-ad58-46b9-97b9-90200f0d1428 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:16.321512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b436b6-ad58-46b9-97b9-90200f0d1428 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:16.321784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b436b6-ad58-46b9-97b9-90200f0d1428 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:16.334969 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b436b6-ad58-46b9-97b9-90200f0d1428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:16.341335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-26b436b6-ad58-46b9-97b9-90200f0d1428 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:16.341335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:16.341335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:16.341335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:16.341335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:16.341335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:16.341335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:16.341335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:16.341335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:16.341335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:16.341335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:16.341335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:16.341335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:16.341335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:16.341335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:16.341335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:16.342250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:16.342250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:16.342250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:16.342250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:16.342250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:16.342250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:16.342250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:16.342250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:16.342250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:16.342250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:16.342250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:16.342250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:16.342250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:16.342250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:16.342250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:16.342250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:16.343026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:16.343026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:16.343026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:16.343026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:16.343026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:16.343026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:16.343026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:16.343026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:16.343026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:16.343026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:16.343026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:16.343026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:16.343026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 57/123] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:57:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:16.344731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b436b6-ad58-46b9-97b9-90200f0d1428 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:16.373681 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60561424-cb23-4244-8508-ff98b063c22b None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:16.418287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aed966-dc43-4e6e-8737-858577211995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:16.418687 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aed966-dc43-4e6e-8737-858577211995 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:16.418973 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aed966-dc43-4e6e-8737-858577211995 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:16.419260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aed966-dc43-4e6e-8737-858577211995 None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:16.420722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aed966-dc43-4e6e-8737-858577211995 None admin] RBAC: Authorizing `identity:get_project(project_id=cc8867738cf24c5db418fa4bd4cd2c01)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:16.422032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:16.422221 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aed966-dc43-4e6e-8737-858577211995 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:16.424140 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 67/124] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:16 2022] GET /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:16.424956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aed966-dc43-4e6e-8737-858577211995 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:16.498022 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b436b6-ad58-46b9-97b9-90200f0d1428 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:16.539691 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef4ab6-50b4-47a8-a684-09121aa23c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:16.540356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef4ab6-50b4-47a8-a684-09121aa23c82 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:16.540621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef4ab6-50b4-47a8-a684-09121aa23c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:16.540800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef4ab6-50b4-47a8-a684-09121aa23c82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:16.554454 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef4ab6-50b4-47a8-a684-09121aa23c82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:16.555709 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aed966-dc43-4e6e-8737-858577211995 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:16.560056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-7fef4ab6-50b4-47a8-a684-09121aa23c82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:16.560056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:16.560056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:16.560056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:16.560056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:16.560056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:16.560056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:16.560056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:16.560056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:16.560056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:16.560056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:16.560056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:16.560056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:16.560056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:16.560056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:16.560056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:16.561001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:16.561001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:16.561001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:16.561001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:16.561001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:16.561001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:16.561001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:16.561001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:16.561001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:16.561001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:16.561001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:16.561001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:16.561001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:16.561001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:16.561001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:16.561001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:16.561784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:16.561784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:16.561784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:16.561784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:16.561784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:16.561784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:16.561784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:16.561784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:16.561784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:16.561784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:16.561784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:16.561784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:16.561784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 58/125] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:16.562426 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef4ab6-50b4-47a8-a684-09121aa23c82 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:16.598806 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:16.604806 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5a84d-8063-432e-a089-7b36f010193f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:16.605817 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5a84d-8063-432e-a089-7b36f010193f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:16.606104 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5a84d-8063-432e-a089-7b36f010193f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:16.606381 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5a84d-8063-432e-a089-7b36f010193f None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:16.607612 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5a84d-8063-432e-a089-7b36f010193f None admin] RBAC: Authorizing `identity:get_project(project_id=cc8867738cf24c5db418fa4bd4cd2c01)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:16.609144 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5a84d-8063-432e-a089-7b36f010193f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:16.611105 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 68/126] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:16 2022] GET /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:16.611656 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5a84d-8063-432e-a089-7b36f010193f None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:16.695906 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef4ab6-50b4-47a8-a684-09121aa23c82 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:16.715227 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:16.730856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5a84d-8063-432e-a089-7b36f010193f None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:16.743129 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74166be7-0da0-474b-9a58-b7642259a86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:16.744246 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74166be7-0da0-474b-9a58-b7642259a86f None admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:16.744729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74166be7-0da0-474b-9a58-b7642259a86f None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:16.745872 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74166be7-0da0-474b-9a58-b7642259a86f None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01/users/d87f64f8b42c4884ba624a882478570d/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:16.748824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74166be7-0da0-474b-9a58-b7642259a86f None admin] RBAC: Authorizing `identity:create_grant(project_id=cc8867738cf24c5db418fa4bd4cd2c01, user_id=d87f64f8b42c4884ba624a882478570d, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:16.751430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74166be7-0da0-474b-9a58-b7642259a86f None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:16.769874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 59/127] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:57:16 2022] PUT /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01/users/d87f64f8b42c4884ba624a882478570d/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:57:16.771013 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74166be7-0da0-474b-9a58-b7642259a86f None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:16.799541 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b08b28-a975-4510-b048-99cb65e472f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:16.800901 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b08b28-a975-4510-b048-99cb65e472f9 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:16.801419 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b08b28-a975-4510-b048-99cb65e472f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:16.802444 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b08b28-a975-4510-b048-99cb65e472f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:16.803879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b08b28-a975-4510-b048-99cb65e472f9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:16.804945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b08b28-a975-4510-b048-99cb65e472f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:16.808261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b08b28-a975-4510-b048-99cb65e472f9 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:16.823222 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 69/128] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:57:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:16.824574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b08b28-a975-4510-b048-99cb65e472f9 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:16.906417 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74166be7-0da0-474b-9a58-b7642259a86f None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:16.931712 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:16.965497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204fd49-c295-42ef-828d-f618eb88f073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:16.966140 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204fd49-c295-42ef-828d-f618eb88f073 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:16.966526 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204fd49-c295-42ef-828d-f618eb88f073 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:16.966701 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204fd49-c295-42ef-828d-f618eb88f073 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:16.967522 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204fd49-c295-42ef-828d-f618eb88f073 None admin] RBAC: Adding query filter params (role.id=46b7af1a00374c1989c960f2be43481e, scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01, user.id=7699f51aaaca4af68243d0c6c54fc93d) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:16.967834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204fd49-c295-42ef-828d-f618eb88f073 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:16.969526 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204fd49-c295-42ef-828d-f618eb88f073 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:16.977894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 60/129] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:57:16 2022] GET /identity/v3/role_assignments?user.id=7699f51aaaca4af68243d0c6c54fc93d&scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01&role.id=46b7af1a00374c1989c960f2be43481e => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:16.986122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204fd49-c295-42ef-828d-f618eb88f073 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:17.029634 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b08b28-a975-4510-b048-99cb65e472f9 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:17.047700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:17.100354 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4395db-2019-41ff-a758-e8d188120793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:17.101964 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4395db-2019-41ff-a758-e8d188120793 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:17.102087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4395db-2019-41ff-a758-e8d188120793 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:17.102599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4395db-2019-41ff-a758-e8d188120793 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:17.104142 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4395db-2019-41ff-a758-e8d188120793 None admin] RBAC: Adding query filter params (name=member) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:17.105009 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4395db-2019-41ff-a758-e8d188120793 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:17.107813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4395db-2019-41ff-a758-e8d188120793 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:17.131218 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 70/130] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:17.132212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4395db-2019-41ff-a758-e8d188120793 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:17.178990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204fd49-c295-42ef-828d-f618eb88f073 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:17.240832 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80c636-54b8-42fd-84dc-fdc720ae6d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:17.241526 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80c636-54b8-42fd-84dc-fdc720ae6d53 None admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:17.241835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80c636-54b8-42fd-84dc-fdc720ae6d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:17.243575 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80c636-54b8-42fd-84dc-fdc720ae6d53 None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01/users/b035f76541b24134b8802d5327d7615c/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:17.248143 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80c636-54b8-42fd-84dc-fdc720ae6d53 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc8867738cf24c5db418fa4bd4cd2c01, user_id=b035f76541b24134b8802d5327d7615c, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:17.250015 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80c636-54b8-42fd-84dc-fdc720ae6d53 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:17.276753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 61/131] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:57:16 2022] PUT /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01/users/b035f76541b24134b8802d5327d7615c/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:57:17.277724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80c636-54b8-42fd-84dc-fdc720ae6d53 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:17.330431 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4395db-2019-41ff-a758-e8d188120793 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:17.366252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:17.389643 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a841ee8-7d6c-4f58-8a29-a6fec9394873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:17.390336 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a841ee8-7d6c-4f58-8a29-a6fec9394873 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:17.390647 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a841ee8-7d6c-4f58-8a29-a6fec9394873 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:17.390928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a841ee8-7d6c-4f58-8a29-a6fec9394873 None admin] PATH_INFO: `/v3/users/d87f64f8b42c4884ba624a882478570d` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:17.393471 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a841ee8-7d6c-4f58-8a29-a6fec9394873 None admin] RBAC: Authorizing `identity:get_user(user_id=d87f64f8b42c4884ba624a882478570d)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:17.395170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a841ee8-7d6c-4f58-8a29-a6fec9394873 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:17.398090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 71/132] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:17 2022] GET /identity/v3/users/d87f64f8b42c4884ba624a882478570d => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:17.399646 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166e2ff-6749-411e-a175-5b6ab030ba19 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:17.399974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166e2ff-6749-411e-a175-5b6ab030ba19 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:17.400219 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166e2ff-6749-411e-a175-5b6ab030ba19 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:17.401971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 72/133] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:17.408737 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166e2ff-6749-411e-a175-5b6ab030ba19 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:17.536539 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80c636-54b8-42fd-84dc-fdc720ae6d53 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:17.540965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:17.592067 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee91b72-6eee-4fa2-90cd-77c82dcefb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:17.592802 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee91b72-6eee-4fa2-90cd-77c82dcefb70 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:17.593090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee91b72-6eee-4fa2-90cd-77c82dcefb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:17.593366 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee91b72-6eee-4fa2-90cd-77c82dcefb70 None admin] PATH_INFO: `/v3/users/d87f64f8b42c4884ba624a882478570d` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:17.594600 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee91b72-6eee-4fa2-90cd-77c82dcefb70 None admin] RBAC: Authorizing `identity:get_user(user_id=d87f64f8b42c4884ba624a882478570d)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:17.595374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166e2ff-6749-411e-a175-5b6ab030ba19 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:17.596064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee91b72-6eee-4fa2-90cd-77c82dcefb70 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:17.598418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 62/134] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:17 2022] GET /identity/v3/users/d87f64f8b42c4884ba624a882478570d => generated 275 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:17.605922 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a20002-f45e-441d-894f-5c07c8ac8313 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:17.606205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a20002-f45e-441d-894f-5c07c8ac8313 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:17.606487 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a20002-f45e-441d-894f-5c07c8ac8313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:17.630352 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-70a20002-f45e-441d-894f-5c07c8ac8313 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:17.631680 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-70a20002-f45e-441d-894f-5c07c8ac8313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:17.651956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3bf26-a92e-4a53-8f15-823129a97df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:17.652695 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3bf26-a92e-4a53-8f15-823129a97df8 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:17.652977 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3bf26-a92e-4a53-8f15-823129a97df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:17.653285 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3bf26-a92e-4a53-8f15-823129a97df8 None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:17.654679 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3bf26-a92e-4a53-8f15-823129a97df8 None admin] RBAC: Authorizing `identity:get_project(project_id=cc8867738cf24c5db418fa4bd4cd2c01)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:17.656346 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3bf26-a92e-4a53-8f15-823129a97df8 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:17.658689 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 73/135] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:17 2022] GET /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:17.659411 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3bf26-a92e-4a53-8f15-823129a97df8 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:17.757580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:17.772613 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 63/136] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:17.774094 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a20002-f45e-441d-894f-5c07c8ac8313 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:17.837674 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3bf26-a92e-4a53-8f15-823129a97df8 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:17.923057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c6ce3-becc-4c3e-82d8-0828fb5ddd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:17.924124 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c6ce3-becc-4c3e-82d8-0828fb5ddd83 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:17.924647 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c6ce3-becc-4c3e-82d8-0828fb5ddd83 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:17.925115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c6ce3-becc-4c3e-82d8-0828fb5ddd83 None admin] PATH_INFO: `/v3/projects/ba14d307309e47ddb630ddc0ae0af22b` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:17.931577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c6ce3-becc-4c3e-82d8-0828fb5ddd83 None admin] RBAC: Authorizing `identity:get_project(project_id=ba14d307309e47ddb630ddc0ae0af22b)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:17.934382 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c6ce3-becc-4c3e-82d8-0828fb5ddd83 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:17.937553 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 74/137] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:17 2022] GET /identity/v3/projects/ba14d307309e47ddb630ddc0ae0af22b => generated 312 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:17.941467 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c6ce3-becc-4c3e-82d8-0828fb5ddd83 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:17.971399 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:17.981113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a20002-f45e-441d-894f-5c07c8ac8313 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:18.089808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c6ce3-becc-4c3e-82d8-0828fb5ddd83 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:18.090160 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25dc125-c4b4-48a9-a42e-ec4a5508f578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:18.090777 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25dc125-c4b4-48a9-a42e-ec4a5508f578 None admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:18.091061 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25dc125-c4b4-48a9-a42e-ec4a5508f578 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:18.091341 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25dc125-c4b4-48a9-a42e-ec4a5508f578 None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01/users/d87f64f8b42c4884ba624a882478570d/roles/e880d7a2b8c74bb1beb5015723a4e3d7` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:18.093605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25dc125-c4b4-48a9-a42e-ec4a5508f578 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc8867738cf24c5db418fa4bd4cd2c01, user_id=d87f64f8b42c4884ba624a882478570d, role_id=e880d7a2b8c74bb1beb5015723a4e3d7)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:18.095062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25dc125-c4b4-48a9-a42e-ec4a5508f578 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:18.118683 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:18.127887 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 64/138] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:57:17 2022] PUT /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01/users/d87f64f8b42c4884ba624a882478570d/roles/e880d7a2b8c74bb1beb5015723a4e3d7 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:57:18.128609 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25dc125-c4b4-48a9-a42e-ec4a5508f578 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:18.142156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f29c61-6080-4184-b424-0d2dec08a433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:18.143005 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f29c61-6080-4184-b424-0d2dec08a433 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:18.143294 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f29c61-6080-4184-b424-0d2dec08a433 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:18.143574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f29c61-6080-4184-b424-0d2dec08a433 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:18.145328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f29c61-6080-4184-b424-0d2dec08a433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:18.147024 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f29c61-6080-4184-b424-0d2dec08a433 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:18.149281 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 75/139] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:57:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:18.156545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f29c61-6080-4184-b424-0d2dec08a433 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:18.361144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25dc125-c4b4-48a9-a42e-ec4a5508f578 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:18.391541 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f29c61-6080-4184-b424-0d2dec08a433 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:18.458816 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8344891-3f00-48cf-b53a-bd514275f302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:18.460113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8344891-3f00-48cf-b53a-bd514275f302 None admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:18.460743 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8344891-3f00-48cf-b53a-bd514275f302 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:18.461251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8344891-3f00-48cf-b53a-bd514275f302 None admin] PATH_INFO: `/v3/projects/ba14d307309e47ddb630ddc0ae0af22b/users/d87f64f8b42c4884ba624a882478570d/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:18.466761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8344891-3f00-48cf-b53a-bd514275f302 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba14d307309e47ddb630ddc0ae0af22b, user_id=d87f64f8b42c4884ba624a882478570d, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:18.468415 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8344891-3f00-48cf-b53a-bd514275f302 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:18.489031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce2873-11cd-4e65-83b5-011484b9cf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:18.489729 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce2873-11cd-4e65-83b5-011484b9cf18 None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:18.490179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce2873-11cd-4e65-83b5-011484b9cf18 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:18.490419 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce2873-11cd-4e65-83b5-011484b9cf18 None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:18.491129 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 65/140] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:57:18 2022] PUT /identity/v3/projects/ba14d307309e47ddb630ddc0ae0af22b/users/d87f64f8b42c4884ba624a882478570d/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:57:18.492838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce2873-11cd-4e65-83b5-011484b9cf18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:18.494708 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce2873-11cd-4e65-83b5-011484b9cf18 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:18.535786 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 76/141] 10.222.0.93 () {60 vars in 1132 bytes} [Mon Aug 1 14:57:18 2022] POST /identity/v3/users => generated 283 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:57:19.493226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e0ab1-b90a-4bf3-81d9-4169387eac59 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:19.494257 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e0ab1-b90a-4bf3-81d9-4169387eac59 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:19.494795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e0ab1-b90a-4bf3-81d9-4169387eac59 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:19.496694 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 66/142] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:19.503720 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b75498-1353-4fd1-8c6c-7ef3fb578427 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:19.504366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b75498-1353-4fd1-8c6c-7ef3fb578427 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:19.504366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b75498-1353-4fd1-8c6c-7ef3fb578427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:19.534103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-31b75498-1353-4fd1-8c6c-7ef3fb578427 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:19.535328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-31b75498-1353-4fd1-8c6c-7ef3fb578427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:19.593274 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:19.615765 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 77/143] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:19.651246 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e0ab1-b90a-4bf3-81d9-4169387eac59 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:19.673110 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:19.706832 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289530f1-8178-4e71-939c-5975ae3c1b6a None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:19.707139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289530f1-8178-4e71-939c-5975ae3c1b6a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:19.707391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289530f1-8178-4e71-939c-5975ae3c1b6a None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:19.708983 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 78/144] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:57:19.714705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e6d7b-a487-427b-8c36-f2fb31978aae None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:19.715138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e6d7b-a487-427b-8c36-f2fb31978aae None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:19.715418 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e6d7b-a487-427b-8c36-f2fb31978aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:19.747707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-411e6d7b-a487-427b-8c36-f2fb31978aae None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:19.748904 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-411e6d7b-a487-427b-8c36-f2fb31978aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:19.773904 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e0ab1-b90a-4bf3-81d9-4169387eac59 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:19.814355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff98b4a-6d51-4983-b44b-37a78f7ecdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:19.815069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff98b4a-6d51-4983-b44b-37a78f7ecdfd None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:19.815349 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff98b4a-6d51-4983-b44b-37a78f7ecdfd None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:19.815666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff98b4a-6d51-4983-b44b-37a78f7ecdfd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:19.827446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff98b4a-6d51-4983-b44b-37a78f7ecdfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:19.832933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-eff98b4a-6d51-4983-b44b-37a78f7ecdfd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:19.832933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:19.832933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:19.832933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:19.832933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:19.832933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:19.832933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:19.832933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:19.832933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:19.832933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:19.832933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:19.832933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:19.832933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:19.832933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:19.832933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:19.832933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:19.833952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:19.833952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:19.833952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:19.833952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:19.833952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:19.833952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:19.833952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:19.833952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:19.833952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:19.833952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:19.833952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:19.833952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:19.833952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:19.833952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:19.833952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:19.833952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:19.834730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:19.834730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:19.834730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:19.834730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:19.834730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:19.834730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:19.834730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:19.834730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:19.834730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:19.834730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:19.834730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:19.834730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:19.834730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 67/145] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:19.843740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff98b4a-6d51-4983-b44b-37a78f7ecdfd None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:19.900247 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 79/146] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:57:19.944445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e6d7b-a487-427b-8c36-f2fb31978aae None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:19.959596 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:19.985334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff98b4a-6d51-4983-b44b-37a78f7ecdfd None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:20.025854 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d76ed-2a08-4f50-b2f9-93a5c06c7130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:20.026385 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d76ed-2a08-4f50-b2f9-93a5c06c7130 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:20.026662 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d76ed-2a08-4f50-b2f9-93a5c06c7130 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:20.026935 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d76ed-2a08-4f50-b2f9-93a5c06c7130 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:20.027705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d76ed-2a08-4f50-b2f9-93a5c06c7130 None admin] RBAC: Adding query filter params (name=member) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:20.028003 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d76ed-2a08-4f50-b2f9-93a5c06c7130 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:20.029536 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d76ed-2a08-4f50-b2f9-93a5c06c7130 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:20.040020 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 68/147] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:20.045720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d76ed-2a08-4f50-b2f9-93a5c06c7130 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:20.151141 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e6d7b-a487-427b-8c36-f2fb31978aae None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:20.169216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d76ed-2a08-4f50-b2f9-93a5c06c7130 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:20.175345 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:20.208675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35db95-5a31-4b85-b22f-458855771c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:20.209386 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35db95-5a31-4b85-b22f-458855771c2a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:20.209678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35db95-5a31-4b85-b22f-458855771c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:20.209964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35db95-5a31-4b85-b22f-458855771c2a None admin] PATH_INFO: `/v3/users/d87f64f8b42c4884ba624a882478570d` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:20.211769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed35db95-5a31-4b85-b22f-458855771c2a None admin] RBAC: Authorizing `identity:get_user(user_id=d87f64f8b42c4884ba624a882478570d)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:20.213372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed35db95-5a31-4b85-b22f-458855771c2a None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:20.215622 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 69/148] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:20 2022] GET /identity/v3/users/d87f64f8b42c4884ba624a882478570d => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:20.221143 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35db95-5a31-4b85-b22f-458855771c2a None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:20.222816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec9251-2bcc-4065-bae2-0a8fd1bbfeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:20.223485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec9251-2bcc-4065-bae2-0a8fd1bbfeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:20.223792 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec9251-2bcc-4065-bae2-0a8fd1bbfeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:20.224147 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec9251-2bcc-4065-bae2-0a8fd1bbfeb4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:20.231244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:20.250857 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec9251-2bcc-4065-bae2-0a8fd1bbfeb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:20.256264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-c3ec9251-2bcc-4065-bae2-0a8fd1bbfeb4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:57:20.256264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:20.256264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:20.256264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:20.256264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:20.256264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:20.256264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:20.256264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:20.256264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:20.256264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:20.256264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:20.256264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:20.256264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:20.256264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:20.256264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:20.256264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:20.257325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:20.257325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:20.257325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:20.257325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:20.257325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:20.257325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:20.257325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:20.257325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:20.257325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:20.257325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:20.257325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:20.257325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:20.257325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:20.257325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:20.257325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:20.257325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:20.258649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:20.258649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:20.258649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:20.258649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:20.258649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:20.258649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:20.258649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:20.258649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:20.258649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:20.258649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:20.258649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:57:20.258649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:20.258649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 80/149] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:57:20.264675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec9251-2bcc-4065-bae2-0a8fd1bbfeb4 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:20.337761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35db95-5a31-4b85-b22f-458855771c2a None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:20.380113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c13c2-40a8-4573-ae0f-639a97b11672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:20.380909 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c13c2-40a8-4573-ae0f-639a97b11672 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:20.381210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c13c2-40a8-4573-ae0f-639a97b11672 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:20.381645 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c13c2-40a8-4573-ae0f-639a97b11672 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:20.392858 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec9251-2bcc-4065-bae2-0a8fd1bbfeb4 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:20.437878 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021b03f-29f1-4405-8efc-386e2b956c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:20.438529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021b03f-29f1-4405-8efc-386e2b956c76 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:20.438820 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021b03f-29f1-4405-8efc-386e2b956c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:20.439326 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021b03f-29f1-4405-8efc-386e2b956c76 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:20.440401 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021b03f-29f1-4405-8efc-386e2b956c76 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:20.440902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021b03f-29f1-4405-8efc-386e2b956c76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:20.442632 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021b03f-29f1-4405-8efc-386e2b956c76 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:20.458805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 81/150] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:20.465009 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021b03f-29f1-4405-8efc-386e2b956c76 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:20.478788 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c13c2-40a8-4573-ae0f-639a97b11672 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:20.480683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c13c2-40a8-4573-ae0f-639a97b11672 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:20.492701 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:20.519448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:20.605881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021b03f-29f1-4405-8efc-386e2b956c76 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:20.620231 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 70/151] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:57:20.626443 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c13c2-40a8-4573-ae0f-639a97b11672 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:20.645115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719802d1-172c-4895-b3b7-e5be0a4f4a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:20.645769 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719802d1-172c-4895-b3b7-e5be0a4f4a17 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:20.646114 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719802d1-172c-4895-b3b7-e5be0a4f4a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:20.646393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719802d1-172c-4895-b3b7-e5be0a4f4a17 None admin] PATH_INFO: `/v3/users/7699f51aaaca4af68243d0c6c54fc93d` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:20.647850 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719802d1-172c-4895-b3b7-e5be0a4f4a17 None admin] RBAC: Authorizing `identity:get_user(user_id=7699f51aaaca4af68243d0c6c54fc93d)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:20.649413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719802d1-172c-4895-b3b7-e5be0a4f4a17 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:20.653947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 82/152] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:20 2022] GET /identity/v3/users/7699f51aaaca4af68243d0c6c54fc93d => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:20.657644 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719802d1-172c-4895-b3b7-e5be0a4f4a17 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:20.746762 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c13c2-40a8-4573-ae0f-639a97b11672 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:20.756010 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:20.788510 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843a854-90f4-4365-89b6-bed5367fe3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:20.789212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843a854-90f4-4365-89b6-bed5367fe3fc None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:20.789490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843a854-90f4-4365-89b6-bed5367fe3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:20.789863 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843a854-90f4-4365-89b6-bed5367fe3fc None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:20.791143 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843a854-90f4-4365-89b6-bed5367fe3fc None admin] RBAC: Authorizing `identity:get_project(project_id=cc8867738cf24c5db418fa4bd4cd2c01)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:20.792637 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843a854-90f4-4365-89b6-bed5367fe3fc None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:20.794538 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 71/153] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:20 2022] GET /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:20.797381 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719802d1-172c-4895-b3b7-e5be0a4f4a17 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:20.799333 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:20.802001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843a854-90f4-4365-89b6-bed5367fe3fc None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:20.840057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccafe6-5130-4273-b8ae-91ce589fc811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:20.840756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccafe6-5130-4273-b8ae-91ce589fc811 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:20.841041 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccafe6-5130-4273-b8ae-91ce589fc811 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:20.841316 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccafe6-5130-4273-b8ae-91ce589fc811 None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:20.842798 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ccafe6-5130-4273-b8ae-91ce589fc811 None admin] RBAC: Authorizing `identity:get_project(project_id=cc8867738cf24c5db418fa4bd4cd2c01)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:20.844452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ccafe6-5130-4273-b8ae-91ce589fc811 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:20.851466 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 83/154] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:20 2022] GET /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:20.852896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccafe6-5130-4273-b8ae-91ce589fc811 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:20.930560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843a854-90f4-4365-89b6-bed5367fe3fc None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:20.974936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d58a0-2c95-4ee0-8ffa-e7d9754cae97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:20.975844 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d58a0-2c95-4ee0-8ffa-e7d9754cae97 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:20.976327 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d58a0-2c95-4ee0-8ffa-e7d9754cae97 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:20.976783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d58a0-2c95-4ee0-8ffa-e7d9754cae97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:20.977863 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d58a0-2c95-4ee0-8ffa-e7d9754cae97 None admin] RBAC: Adding query filter params (role.id=c65b8ad9ade94760af715b3899bdd799, scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01, user.id=d87f64f8b42c4884ba624a882478570d) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:20.978488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d58a0-2c95-4ee0-8ffa-e7d9754cae97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:20.980188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d58a0-2c95-4ee0-8ffa-e7d9754cae97 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:20.986292 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccafe6-5130-4273-b8ae-91ce589fc811 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:20.988019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 72/155] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:57:20 2022] GET /identity/v3/role_assignments?user.id=d87f64f8b42c4884ba624a882478570d&scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01&role.id=c65b8ad9ade94760af715b3899bdd799 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:21.033803 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f3838-e2c7-426d-8e22-f1d26ac3c553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:21.034517 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f3838-e2c7-426d-8e22-f1d26ac3c553 None admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:21.034800 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f3838-e2c7-426d-8e22-f1d26ac3c553 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:21.035086 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f3838-e2c7-426d-8e22-f1d26ac3c553 None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01/users/7699f51aaaca4af68243d0c6c54fc93d/roles/46b7af1a00374c1989c960f2be43481e` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:21.037338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f3838-e2c7-426d-8e22-f1d26ac3c553 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc8867738cf24c5db418fa4bd4cd2c01, user_id=7699f51aaaca4af68243d0c6c54fc93d, role_id=46b7af1a00374c1989c960f2be43481e)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:21.038920 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f3838-e2c7-426d-8e22-f1d26ac3c553 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:21.056447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 84/156] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:57:20 2022] PUT /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01/users/7699f51aaaca4af68243d0c6c54fc93d/roles/46b7af1a00374c1989c960f2be43481e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:57:21.437001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29485168-c233-456d-a65e-58a77580ffeb None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:21.437580 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29485168-c233-456d-a65e-58a77580ffeb None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:21.438065 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29485168-c233-456d-a65e-58a77580ffeb None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:21.439619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 73/157] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:21.446049 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62564a-20f7-4395-acaf-474bafce97ae None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:21.446333 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62564a-20f7-4395-acaf-474bafce97ae None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:21.446609 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62564a-20f7-4395-acaf-474bafce97ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:21.472995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-de62564a-20f7-4395-acaf-474bafce97ae None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:21.474742 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-de62564a-20f7-4395-acaf-474bafce97ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:21.522350 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:21.579504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 85/158] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:21.612543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29485168-c233-456d-a65e-58a77580ffeb None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:21.670231 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:21.742722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29485168-c233-456d-a65e-58a77580ffeb None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:21.780301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6a690-69d3-4f2d-be2e-1c6047323881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:21.781058 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6a690-69d3-4f2d-be2e-1c6047323881 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:21.781337 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6a690-69d3-4f2d-be2e-1c6047323881 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:21.781609 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6a690-69d3-4f2d-be2e-1c6047323881 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:21.792874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6a690-69d3-4f2d-be2e-1c6047323881 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:21.797895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-93a6a690-69d3-4f2d-be2e-1c6047323881 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:57:21.797895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:21.797895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:21.797895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:21.797895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:21.797895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:21.797895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:21.797895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:21.797895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:21.797895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:21.797895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:21.797895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:21.797895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:21.797895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:21.797895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:21.797895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:21.799989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:21.799989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:21.799989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:21.799989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:21.799989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:21.799989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:21.799989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:21.799989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:21.799989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:21.799989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:21.799989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:21.799989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:21.799989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:21.799989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:21.799989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:21.799989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:21.802008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:21.802008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:21.802008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:21.802008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:21.802008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:21.802008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:21.802008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:21.802008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:21.802008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:21.802008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:21.802008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:57:21.802008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:21.802008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 74/159] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:57:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:21.806846 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62564a-20f7-4395-acaf-474bafce97ae None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:21.914195 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62564a-20f7-4395-acaf-474bafce97ae None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:21.949986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a467d7-c7a7-43df-ab06-3beac660b80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:21.950605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a467d7-c7a7-43df-ab06-3beac660b80f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:21.950881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a467d7-c7a7-43df-ab06-3beac660b80f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:21.951150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a467d7-c7a7-43df-ab06-3beac660b80f None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:21.951866 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a467d7-c7a7-43df-ab06-3beac660b80f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:21.952161 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a467d7-c7a7-43df-ab06-3beac660b80f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:21.953581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a467d7-c7a7-43df-ab06-3beac660b80f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:21.963151 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 86/160] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:21.968998 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6a690-69d3-4f2d-be2e-1c6047323881 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:22.076989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:22.079285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6a690-69d3-4f2d-be2e-1c6047323881 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:22.114806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84602c8-1e44-4097-b2ae-f95c2a062aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:22.115450 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84602c8-1e44-4097-b2ae-f95c2a062aca None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:22.115726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84602c8-1e44-4097-b2ae-f95c2a062aca None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:22.115992 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84602c8-1e44-4097-b2ae-f95c2a062aca None admin] PATH_INFO: `/v3/users/b035f76541b24134b8802d5327d7615c` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:22.117715 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84602c8-1e44-4097-b2ae-f95c2a062aca None admin] RBAC: Authorizing `identity:get_user(user_id=b035f76541b24134b8802d5327d7615c)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:22.119185 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84602c8-1e44-4097-b2ae-f95c2a062aca None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:22.120847 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 75/161] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:21 2022] GET /identity/v3/users/b035f76541b24134b8802d5327d7615c => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:22.126749 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a467d7-c7a7-43df-ab06-3beac660b80f None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:22.152990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:22.232434 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a467d7-c7a7-43df-ab06-3beac660b80f None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:22.271248 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d60280-8f7d-4295-a093-544867acf386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:22.271869 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d60280-8f7d-4295-a093-544867acf386 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:22.272142 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d60280-8f7d-4295-a093-544867acf386 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:22.272409 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d60280-8f7d-4295-a093-544867acf386 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:22.311289 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747ce8d-599f-4804-b74d-0b68f7cae998 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:22.311593 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747ce8d-599f-4804-b74d-0b68f7cae998 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:22.311841 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747ce8d-599f-4804-b74d-0b68f7cae998 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:22.313535 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 76/162] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:22.318564 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae727629-7288-4902-8635-4e9ff87e16b0 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:22.318835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae727629-7288-4902-8635-4e9ff87e16b0 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:22.319107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae727629-7288-4902-8635-4e9ff87e16b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:22.346714 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-ae727629-7288-4902-8635-4e9ff87e16b0 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:22.357317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ae727629-7288-4902-8635-4e9ff87e16b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:22.374290 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d60280-8f7d-4295-a093-544867acf386 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:22.375831 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d60280-8f7d-4295-a093-544867acf386 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:22.416848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:22.495492 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 87/163] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:57:22.501464 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d60280-8f7d-4295-a093-544867acf386 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:22.578530 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 77/164] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:22.612278 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d60280-8f7d-4295-a093-544867acf386 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:22.629620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae727629-7288-4902-8635-4e9ff87e16b0 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:22.651423 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f763fd6-e367-4d2b-95b9-8edf2ed85279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:22.652360 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f763fd6-e367-4d2b-95b9-8edf2ed85279 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:22.652817 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f763fd6-e367-4d2b-95b9-8edf2ed85279 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:22.653215 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f763fd6-e367-4d2b-95b9-8edf2ed85279 None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:22.654691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f763fd6-e367-4d2b-95b9-8edf2ed85279 None admin] RBAC: Authorizing `identity:get_project(project_id=cc8867738cf24c5db418fa4bd4cd2c01)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:22.656380 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f763fd6-e367-4d2b-95b9-8edf2ed85279 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:22.658585 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 88/165] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:22 2022] GET /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:22.660710 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:22.678138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f763fd6-e367-4d2b-95b9-8edf2ed85279 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:22.796693 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f763fd6-e367-4d2b-95b9-8edf2ed85279 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:22.834676 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2edfd-6c16-44af-a1e4-dbb1c75e236b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:22.836172 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2edfd-6c16-44af-a1e4-dbb1c75e236b None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:22.836745 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2edfd-6c16-44af-a1e4-dbb1c75e236b None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:22.837199 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2edfd-6c16-44af-a1e4-dbb1c75e236b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:22.838193 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2edfd-6c16-44af-a1e4-dbb1c75e236b None admin] RBAC: Adding query filter params (role.id=5456887a306649bf80a9ea7bd4282e97, scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01, user.id=b035f76541b24134b8802d5327d7615c) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:22.838700 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2edfd-6c16-44af-a1e4-dbb1c75e236b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:22.840513 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2edfd-6c16-44af-a1e4-dbb1c75e236b None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:22.847518 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 89/166] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:57:22 2022] GET /identity/v3/role_assignments?user.id=b035f76541b24134b8802d5327d7615c&scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:22.859035 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:22.866995 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae727629-7288-4902-8635-4e9ff87e16b0 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:22.908660 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def966b-860e-4713-aa41-97d7a4143c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:22.909344 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def966b-860e-4713-aa41-97d7a4143c17 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:22.909627 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def966b-860e-4713-aa41-97d7a4143c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:22.909902 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def966b-860e-4713-aa41-97d7a4143c17 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:22.922731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def966b-860e-4713-aa41-97d7a4143c17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:22.927824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-3def966b-860e-4713-aa41-97d7a4143c17 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:22.927824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:22.927824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:22.927824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:22.927824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:22.927824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:22.927824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:22.927824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:22.927824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:22.927824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:22.927824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:22.927824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:22.927824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:22.927824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:22.927824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:22.927824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:22.928757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:22.928757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:22.928757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:22.928757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:22.928757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:22.928757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:22.928757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:22.928757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:22.928757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:22.928757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:22.928757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:22.928757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:22.928757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:22.928757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:22.928757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:22.928757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:22.929761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:22.929761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:22.929761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:22.929761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:22.929761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:22.929761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:22.929761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:22.929761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:22.929761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:22.929761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:22.929761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:22.929761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:22.929761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 78/167] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:57:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:22.936939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2edfd-6c16-44af-a1e4-dbb1c75e236b None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:23.006091 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:23.066180 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2edfd-6c16-44af-a1e4-dbb1c75e236b None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:23.131283 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fbe9e-6d3f-4ecb-a11b-a8cb58809282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:23.131866 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fbe9e-6d3f-4ecb-a11b-a8cb58809282 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:23.132145 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fbe9e-6d3f-4ecb-a11b-a8cb58809282 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:23.133091 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fbe9e-6d3f-4ecb-a11b-a8cb58809282 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:23.134052 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809fbe9e-6d3f-4ecb-a11b-a8cb58809282 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:23.134381 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809fbe9e-6d3f-4ecb-a11b-a8cb58809282 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:23.135883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809fbe9e-6d3f-4ecb-a11b-a8cb58809282 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:23.145281 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 90/168] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:57:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:23.151590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def966b-860e-4713-aa41-97d7a4143c17 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:23.270014 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def966b-860e-4713-aa41-97d7a4143c17 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:23.314629 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9ab1b-ee78-4847-a279-5575ecc60a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:23.315315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9ab1b-ee78-4847-a279-5575ecc60a45 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:23.315926 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9ab1b-ee78-4847-a279-5575ecc60a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:23.316201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9ab1b-ee78-4847-a279-5575ecc60a45 None admin] PATH_INFO: `/v3/users/d87f64f8b42c4884ba624a882478570d` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:23.317656 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a9ab1b-ee78-4847-a279-5575ecc60a45 None admin] RBAC: Authorizing `identity:get_user(user_id=d87f64f8b42c4884ba624a882478570d)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:23.319339 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a9ab1b-ee78-4847-a279-5575ecc60a45 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:23.322126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 79/169] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:23 2022] GET /identity/v3/users/d87f64f8b42c4884ba624a882478570d => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:23.328569 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fbe9e-6d3f-4ecb-a11b-a8cb58809282 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:23.368516 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e505927-10a7-445b-9866-dd3d4beaab6d None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:23.368833 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e505927-10a7-445b-9866-dd3d4beaab6d None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:23.369066 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e505927-10a7-445b-9866-dd3d4beaab6d None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:23.370542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 80/170] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:57:23.377222 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc56d7-d093-41bb-8c54-f351acb7dd9a None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:23.377648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc56d7-d093-41bb-8c54-f351acb7dd9a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:23.377939 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc56d7-d093-41bb-8c54-f351acb7dd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:23.417951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-b7bc56d7-d093-41bb-8c54-f351acb7dd9a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:23.419354 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b7bc56d7-d093-41bb-8c54-f351acb7dd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:23.472858 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:23.578298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fbe9e-6d3f-4ecb-a11b-a8cb58809282 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:23.592206 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 81/171] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:57:23.593575 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1049b66-6af9-4407-9625-fb49a38931e5 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:23.593861 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1049b66-6af9-4407-9625-fb49a38931e5 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:23.594136 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1049b66-6af9-4407-9625-fb49a38931e5 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:23.599895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 82/172] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:23.605101 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785055e0-3ea4-4545-a455-c0f2483eb746 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:23.605439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785055e0-3ea4-4545-a455-c0f2483eb746 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:23.605717 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785055e0-3ea4-4545-a455-c0f2483eb746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:23.612252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:23.639619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-785055e0-3ea4-4545-a455-c0f2483eb746 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:23.641047 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-785055e0-3ea4-4545-a455-c0f2483eb746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:23.645045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659ac44-a880-4ebf-aa9d-50be458d3b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:23.646335 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659ac44-a880-4ebf-aa9d-50be458d3b0e None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:23.646808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659ac44-a880-4ebf-aa9d-50be458d3b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:23.647298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659ac44-a880-4ebf-aa9d-50be458d3b0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:23.733490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 83/173] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:23.734475 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785055e0-3ea4-4545-a455-c0f2483eb746 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:23.769825 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659ac44-a880-4ebf-aa9d-50be458d3b0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:23.771824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659ac44-a880-4ebf-aa9d-50be458d3b0e None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:23.844597 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:23.858680 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785055e0-3ea4-4545-a455-c0f2483eb746 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:23.900112 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a50d99-644a-4093-8b15-7a80116bd614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:23.900921 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a50d99-644a-4093-8b15-7a80116bd614 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:23.901220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a50d99-644a-4093-8b15-7a80116bd614 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:23.901436 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a50d99-644a-4093-8b15-7a80116bd614 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:23.913119 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a50d99-644a-4093-8b15-7a80116bd614 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:23.918727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-14a50d99-644a-4093-8b15-7a80116bd614 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:23.918727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:23.918727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:23.918727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:23.918727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:23.918727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:23.918727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:23.918727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:23.918727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:23.918727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:23.918727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:23.918727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:23.918727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:23.918727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:23.918727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:23.918727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:23.919623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:23.919623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:23.919623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:23.919623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:23.919623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:23.919623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:23.919623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:23.919623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:23.919623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:23.919623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:23.919623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:23.919623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:23.919623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:23.919623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:23.919623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:23.919623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:23.920456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:23.920456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:23.920456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:23.920456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:23.920456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:23.920456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:23.920456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:23.920456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:23.920456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:23.920456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:23.920456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:23.920456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:23.920456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 84/174] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:23.921851 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a50d99-644a-4093-8b15-7a80116bd614 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:24.015230 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 91/175] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:57:24.016065 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659ac44-a880-4ebf-aa9d-50be458d3b0e None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:24.044693 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:24.110387 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a50d99-644a-4093-8b15-7a80116bd614 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:24.171287 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c224f-e9cb-4a47-b25b-651936ba84d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:24.172513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c224f-e9cb-4a47-b25b-651936ba84d5 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:24.172800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c224f-e9cb-4a47-b25b-651936ba84d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:24.173075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c224f-e9cb-4a47-b25b-651936ba84d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:24.188001 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659ac44-a880-4ebf-aa9d-50be458d3b0e None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:24.188985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c224f-e9cb-4a47-b25b-651936ba84d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:24.195994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-d53c224f-e9cb-4a47-b25b-651936ba84d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:57:24.195994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:24.195994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:24.195994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:24.195994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:24.195994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:24.195994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:24.195994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:24.195994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:24.195994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:24.195994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:24.195994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:24.195994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:24.195994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:24.195994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:24.195994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:24.196960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:24.196960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:24.196960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:24.196960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:24.196960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:24.196960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:24.196960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:24.196960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:24.196960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:24.196960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:24.196960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:24.196960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:24.196960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:24.196960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:24.196960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:24.196960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:24.197763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:24.197763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:24.197763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:24.197763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:24.197763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:24.197763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:24.197763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:24.197763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:24.197763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:24.197763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:24.197763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:57:24.197763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:24.198386 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 85/176] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:57:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:24.198978 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c224f-e9cb-4a47-b25b-651936ba84d5 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:24.204816 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:24.232346 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e194b8-37bd-4d77-98a7-a7531d70eb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:24.233216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e194b8-37bd-4d77-98a7-a7531d70eb53 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:24.233632 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e194b8-37bd-4d77-98a7-a7531d70eb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:24.233910 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e194b8-37bd-4d77-98a7-a7531d70eb53 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:24.234647 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e194b8-37bd-4d77-98a7-a7531d70eb53 None admin] RBAC: Adding query filter params (name=member) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:24.234950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e194b8-37bd-4d77-98a7-a7531d70eb53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:24.237851 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e194b8-37bd-4d77-98a7-a7531d70eb53 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:24.262059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 92/177] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:24.263779 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eaabec-4078-44fb-ab21-35920d2518c8 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:24.264141 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eaabec-4078-44fb-ab21-35920d2518c8 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:24.264418 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eaabec-4078-44fb-ab21-35920d2518c8 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:24.265850 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 93/178] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:24.266638 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eaabec-4078-44fb-ab21-35920d2518c8 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:24.336098 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c224f-e9cb-4a47-b25b-651936ba84d5 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:24.368357 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:24.374629 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668eceaa-2256-4546-9fbb-6af98302985f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:24.375359 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668eceaa-2256-4546-9fbb-6af98302985f None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:24.375669 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668eceaa-2256-4546-9fbb-6af98302985f None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:24.375987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668eceaa-2256-4546-9fbb-6af98302985f None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:24.377377 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668eceaa-2256-4546-9fbb-6af98302985f None admin] RBAC: Authorizing `identity:get_project(project_id=cc8867738cf24c5db418fa4bd4cd2c01)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:24.378981 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668eceaa-2256-4546-9fbb-6af98302985f None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:24.381211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 86/179] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:24 2022] GET /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:24.387314 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668eceaa-2256-4546-9fbb-6af98302985f None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:24.402759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eaabec-4078-44fb-ab21-35920d2518c8 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:24.444229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ad325-ee2c-44c9-90f6-c4e33c29c810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:24.445464 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ad325-ee2c-44c9-90f6-c4e33c29c810 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:24.445816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ad325-ee2c-44c9-90f6-c4e33c29c810 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:24.446102 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ad325-ee2c-44c9-90f6-c4e33c29c810 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:24.446828 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ad325-ee2c-44c9-90f6-c4e33c29c810 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:24.447133 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ad325-ee2c-44c9-90f6-c4e33c29c810 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:24.448652 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ad325-ee2c-44c9-90f6-c4e33c29c810 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:24.458946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 94/180] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:24.462099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6471686-8e41-497a-98b0-936c2b9c726a None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:24.462432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6471686-8e41-497a-98b0-936c2b9c726a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:24.462661 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6471686-8e41-497a-98b0-936c2b9c726a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:24.483551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:24.487492 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-b6471686-8e41-497a-98b0-936c2b9c726a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:24.488755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b6471686-8e41-497a-98b0-936c2b9c726a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:24.526540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668eceaa-2256-4546-9fbb-6af98302985f None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:24.563352 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b09d0-9de1-4aeb-ae76-ee93ee0ea016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:24.564021 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b09d0-9de1-4aeb-ae76-ee93ee0ea016 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:24.571747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 95/181] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:24.571908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b09d0-9de1-4aeb-ae76-ee93ee0ea016 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:24.571908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b09d0-9de1-4aeb-ae76-ee93ee0ea016 None admin] PATH_INFO: `/v3/users/d87f64f8b42c4884ba624a882478570d` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:24.572062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d437358-cb4b-4b8b-a9b9-555f695361bd None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:24.572062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d437358-cb4b-4b8b-a9b9-555f695361bd None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:24.572062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d437358-cb4b-4b8b-a9b9-555f695361bd None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:24.572062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 96/182] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:24.572062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d437358-cb4b-4b8b-a9b9-555f695361bd None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:24.573476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b09d0-9de1-4aeb-ae76-ee93ee0ea016 None admin] RBAC: Authorizing `identity:get_user(user_id=d87f64f8b42c4884ba624a882478570d)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:24.574970 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b09d0-9de1-4aeb-ae76-ee93ee0ea016 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:24.577296 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 87/183] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:24 2022] GET /identity/v3/users/d87f64f8b42c4884ba624a882478570d => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:24.577807 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b09d0-9de1-4aeb-ae76-ee93ee0ea016 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:24.651574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:24.687786 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d437358-cb4b-4b8b-a9b9-555f695361bd None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:24.693860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b09d0-9de1-4aeb-ae76-ee93ee0ea016 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:24.724697 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e6f70-858e-4118-b0a5-ddfb09af7186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:24.725330 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e6f70-858e-4118-b0a5-ddfb09af7186 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:24.725601 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e6f70-858e-4118-b0a5-ddfb09af7186 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:24.725912 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e6f70-858e-4118-b0a5-ddfb09af7186 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:24.727193 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e6f70-858e-4118-b0a5-ddfb09af7186 None admin] RBAC: Adding query filter params (role.id=e880d7a2b8c74bb1beb5015723a4e3d7, scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01, user.id=d87f64f8b42c4884ba624a882478570d) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:24.727498 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e6f70-858e-4118-b0a5-ddfb09af7186 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:24.729001 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e6f70-858e-4118-b0a5-ddfb09af7186 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:24.730408 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27edf832-631d-4860-aa66-b4a95966ff55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:24.731071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edf832-631d-4860-aa66-b4a95966ff55 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:24.731351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edf832-631d-4860-aa66-b4a95966ff55 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:24.731626 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edf832-631d-4860-aa66-b4a95966ff55 None admin] PATH_INFO: `/v3/users/27be4cdbaaec46c9b44d1f5474b281ae` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:24.735673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 97/184] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:57:24 2022] GET /identity/v3/role_assignments?user.id=d87f64f8b42c4884ba624a882478570d&scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01&role.id=e880d7a2b8c74bb1beb5015723a4e3d7 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:24.737064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ae8a5-4a70-4f08-a2a1-b3c4831c1968 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:24.737346 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ae8a5-4a70-4f08-a2a1-b3c4831c1968 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:24.737615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ae8a5-4a70-4f08-a2a1-b3c4831c1968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:24.745460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:24.750121 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27edf832-631d-4860-aa66-b4a95966ff55 None admin] RBAC: Authorizing `identity:get_user(user_id=27be4cdbaaec46c9b44d1f5474b281ae)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:24.751727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27edf832-631d-4860-aa66-b4a95966ff55 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:24.753616 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 88/185] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:24 2022] GET /identity/v3/users/27be4cdbaaec46c9b44d1f5474b281ae => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:24.754933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c6528-7b0e-4d90-87ad-58d0c2a3b443 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:24.755222 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c6528-7b0e-4d90-87ad-58d0c2a3b443 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:24.755553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c6528-7b0e-4d90-87ad-58d0c2a3b443 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:24.757198 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 89/186] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:24.757773 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c6528-7b0e-4d90-87ad-58d0c2a3b443 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:24.772442 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-b83ae8a5-4a70-4f08-a2a1-b3c4831c1968 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:24.774444 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b83ae8a5-4a70-4f08-a2a1-b3c4831c1968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:24.885880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 98/187] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:24.896803 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ae8a5-4a70-4f08-a2a1-b3c4831c1968 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:24.916811 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c6528-7b0e-4d90-87ad-58d0c2a3b443 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:24.958165 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28220ec-04df-4e8c-bb8e-2048fe3449c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:24.958856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28220ec-04df-4e8c-bb8e-2048fe3449c2 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:24.959139 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28220ec-04df-4e8c-bb8e-2048fe3449c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:24.959649 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28220ec-04df-4e8c-bb8e-2048fe3449c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:24.987164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:25.049377 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ae8a5-4a70-4f08-a2a1-b3c4831c1968 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:25.051990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:25.057087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28220ec-04df-4e8c-bb8e-2048fe3449c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:25.059417 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28220ec-04df-4e8c-bb8e-2048fe3449c2 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:25.097854 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9d8db-2479-43e9-ad41-cfbb230fd048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:25.098940 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9d8db-2479-43e9-ad41-cfbb230fd048 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:25.099454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9d8db-2479-43e9-ad41-cfbb230fd048 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:25.099962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9d8db-2479-43e9-ad41-cfbb230fd048 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:25.102808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9d8db-2479-43e9-ad41-cfbb230fd048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:25.105314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9d8db-2479-43e9-ad41-cfbb230fd048 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:25.108078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 99/188] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:57:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:25.109598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9d8db-2479-43e9-ad41-cfbb230fd048 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:25.200428 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 90/189] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:57:25.201736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81753c7-7ce2-44a0-9ae0-6b10c17281b8 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:25.202031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81753c7-7ce2-44a0-9ae0-6b10c17281b8 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:25.202313 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81753c7-7ce2-44a0-9ae0-6b10c17281b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:25.222183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-c81753c7-7ce2-44a0-9ae0-6b10c17281b8 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:25.223448 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c81753c7-7ce2-44a0-9ae0-6b10c17281b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:25.241386 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9d8db-2479-43e9-ad41-cfbb230fd048 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:25.281773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:25.288369 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87589d9-2382-48cd-a69b-2db6e615264d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:25.289602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87589d9-2382-48cd-a69b-2db6e615264d None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:25.289884 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87589d9-2382-48cd-a69b-2db6e615264d None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:25.290161 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87589d9-2382-48cd-a69b-2db6e615264d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:25.313985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 91/190] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:25.328670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81753c7-7ce2-44a0-9ae0-6b10c17281b8 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:25.346111 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:25.369779 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87589d9-2382-48cd-a69b-2db6e615264d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:25.371339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87589d9-2382-48cd-a69b-2db6e615264d None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:25.449337 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81753c7-7ce2-44a0-9ae0-6b10c17281b8 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:25.502104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be2547-e356-43a5-9d48-8d79058ab1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:25.502775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be2547-e356-43a5-9d48-8d79058ab1df None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:25.505621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be2547-e356-43a5-9d48-8d79058ab1df None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:25.505621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be2547-e356-43a5-9d48-8d79058ab1df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:25.517655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be2547-e356-43a5-9d48-8d79058ab1df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:25.522814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-06be2547-e356-43a5-9d48-8d79058ab1df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:25.522814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:25.522814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:25.522814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:25.522814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:25.522814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:25.522814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:25.522814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:25.522814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:25.522814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:25.522814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:25.522814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:25.522814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:25.522814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:25.522814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:25.522814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:25.524964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:25.524964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:25.524964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:25.524964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:25.524964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:25.524964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:25.524964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:25.524964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:25.524964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:25.524964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:25.524964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:25.524964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:25.524964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:25.524964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:25.524964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:25.524964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:25.526490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:25.526490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:25.526490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:25.526490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:25.526490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:25.526490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:25.526490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:25.526490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:25.526490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:25.526490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:25.526490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:25.526490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:25.527643 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 100/191] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:57:25.527772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 92/192] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:57:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:25.527890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87589d9-2382-48cd-a69b-2db6e615264d None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:25.528009 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be2547-e356-43a5-9d48-8d79058ab1df None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:25.577546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:25.640770 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:25.657631 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be2547-e356-43a5-9d48-8d79058ab1df None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:25.662662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87589d9-2382-48cd-a69b-2db6e615264d None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:25.697888 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842d5a1-a86e-4a2d-8492-41e564be3c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:25.698570 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842d5a1-a86e-4a2d-8492-41e564be3c89 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:25.698849 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842d5a1-a86e-4a2d-8492-41e564be3c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:25.699119 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842d5a1-a86e-4a2d-8492-41e564be3c89 None admin] PATH_INFO: `/v3/projects/ba14d307309e47ddb630ddc0ae0af22b` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:25.700351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842d5a1-a86e-4a2d-8492-41e564be3c89 None admin] RBAC: Authorizing `identity:get_project(project_id=ba14d307309e47ddb630ddc0ae0af22b)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:25.701896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842d5a1-a86e-4a2d-8492-41e564be3c89 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:25.703650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 93/193] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:25 2022] GET /identity/v3/projects/ba14d307309e47ddb630ddc0ae0af22b => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:25.709969 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842d5a1-a86e-4a2d-8492-41e564be3c89 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:25.710432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7060497-0ede-4e4e-8b55-5d3f376731ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:25.711084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7060497-0ede-4e4e-8b55-5d3f376731ed None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:25.711364 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7060497-0ede-4e4e-8b55-5d3f376731ed None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:25.711633 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7060497-0ede-4e4e-8b55-5d3f376731ed None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:25.712340 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7060497-0ede-4e4e-8b55-5d3f376731ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:25.713863 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7060497-0ede-4e4e-8b55-5d3f376731ed None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:25.743989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 101/194] 10.222.0.93 () {60 vars in 1132 bytes} [Mon Aug 1 14:57:25 2022] POST /identity/v3/users => generated 297 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:57:25.743989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7060497-0ede-4e4e-8b55-5d3f376731ed None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:25.883494 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842d5a1-a86e-4a2d-8492-41e564be3c89 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:25.908941 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:25.933498 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7060497-0ede-4e4e-8b55-5d3f376731ed None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:25.961873 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e1fec-67db-45cd-be81-220ef2dbf2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:25.963075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e1fec-67db-45cd-be81-220ef2dbf2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:25.963449 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e1fec-67db-45cd-be81-220ef2dbf2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:25.964774 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e1fec-67db-45cd-be81-220ef2dbf2d3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:25.983302 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e1fec-67db-45cd-be81-220ef2dbf2d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:25.990837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-bc3e1fec-67db-45cd-be81-220ef2dbf2d3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:57:25.990837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:25.990837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:25.990837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:25.990837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:25.990837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:25.990837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:25.990837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:25.990837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:25.990837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:25.990837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:25.990837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:25.990837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:25.990837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:25.990837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:25.990837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:25.991987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:25.991987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:25.991987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:25.991987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:25.991987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:25.991987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:25.991987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:25.991987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:25.991987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:25.991987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:25.991987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:25.991987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:25.991987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:25.991987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:25.991987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:25.991987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:25.992989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:25.992989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:25.992989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:25.992989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:25.992989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:25.992989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:25.992989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:25.992989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:25.992989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:25.992989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:25.992989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:57:25.992989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:25.992989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 94/195] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:25.993665 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e1fec-67db-45cd-be81-220ef2dbf2d3 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:26.014070 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868031d-e8b3-41e4-8285-094c0946af53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:26.014790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868031d-e8b3-41e4-8285-094c0946af53 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:26.015511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868031d-e8b3-41e4-8285-094c0946af53 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:26.015807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868031d-e8b3-41e4-8285-094c0946af53 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:26.016592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868031d-e8b3-41e4-8285-094c0946af53 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:26.017242 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868031d-e8b3-41e4-8285-094c0946af53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:26.019354 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868031d-e8b3-41e4-8285-094c0946af53 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:26.025481 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:26.036580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 102/196] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:57:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:26.036580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868031d-e8b3-41e4-8285-094c0946af53 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:26.204045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e1fec-67db-45cd-be81-220ef2dbf2d3 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:26.223950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868031d-e8b3-41e4-8285-094c0946af53 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:26.295724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de1e8d-74ab-4958-9373-62d3f23efe89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:26.296952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de1e8d-74ab-4958-9373-62d3f23efe89 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:26.297445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de1e8d-74ab-4958-9373-62d3f23efe89 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:26.297958 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de1e8d-74ab-4958-9373-62d3f23efe89 None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:26.300280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de1e8d-74ab-4958-9373-62d3f23efe89 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4838b7b9e4303a80d7610c6bef277)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:26.303217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de1e8d-74ab-4958-9373-62d3f23efe89 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:26.303892 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b8953-cfda-46ef-be73-5646c0dc2cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:26.305475 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b8953-cfda-46ef-be73-5646c0dc2cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:26.306402 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b8953-cfda-46ef-be73-5646c0dc2cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:26.306684 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 95/197] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:25 2022] GET /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277 => generated 302 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:26.307662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b8953-cfda-46ef-be73-5646c0dc2cb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:26.308139 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de1e8d-74ab-4958-9373-62d3f23efe89 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:26.310291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b8953-cfda-46ef-be73-5646c0dc2cb8 None admin] RBAC: Adding query filter params (role.id=c65b8ad9ade94760af715b3899bdd799, scope.project.id=ba14d307309e47ddb630ddc0ae0af22b, user.id=d87f64f8b42c4884ba624a882478570d) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:26.311371 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b8953-cfda-46ef-be73-5646c0dc2cb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:26.317142 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b8953-cfda-46ef-be73-5646c0dc2cb8 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:26.331846 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 103/198] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:57:26 2022] GET /identity/v3/role_assignments?user.id=d87f64f8b42c4884ba624a882478570d&scope.project.id=ba14d307309e47ddb630ddc0ae0af22b&role.id=c65b8ad9ade94760af715b3899bdd799 => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:26.335095 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b8953-cfda-46ef-be73-5646c0dc2cb8 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:26.370561 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:26.491215 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:26.514940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de1e8d-74ab-4958-9373-62d3f23efe89 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:26.564078 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed8f4c-0323-4c37-932f-eccd5368fdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:26.565573 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed8f4c-0323-4c37-932f-eccd5368fdc8 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:26.566233 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed8f4c-0323-4c37-932f-eccd5368fdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:26.567681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed8f4c-0323-4c37-932f-eccd5368fdc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:26.569180 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed8f4c-0323-4c37-932f-eccd5368fdc8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:26.569810 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed8f4c-0323-4c37-932f-eccd5368fdc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:26.572394 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed8f4c-0323-4c37-932f-eccd5368fdc8 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:26.586686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 96/199] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:26.588578 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed8f4c-0323-4c37-932f-eccd5368fdc8 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:26.689623 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b8953-cfda-46ef-be73-5646c0dc2cb8 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:26.753601 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192149a-e4d7-4dd2-a3f0-5055abed754f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:26.754556 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192149a-e4d7-4dd2-a3f0-5055abed754f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:26.755373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192149a-e4d7-4dd2-a3f0-5055abed754f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:26.755659 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192149a-e4d7-4dd2-a3f0-5055abed754f None admin] PATH_INFO: `/v3/users/27be4cdbaaec46c9b44d1f5474b281ae` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:26.758382 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2192149a-e4d7-4dd2-a3f0-5055abed754f None admin] RBAC: Authorizing `identity:get_user(user_id=27be4cdbaaec46c9b44d1f5474b281ae)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:26.761174 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2192149a-e4d7-4dd2-a3f0-5055abed754f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:26.764239 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 104/200] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:26 2022] GET /identity/v3/users/27be4cdbaaec46c9b44d1f5474b281ae => generated 283 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:26.765918 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192149a-e4d7-4dd2-a3f0-5055abed754f None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:26.784957 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed8f4c-0323-4c37-932f-eccd5368fdc8 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:26.865435 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2532fe7-0779-408d-b40e-f130616bec57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:26.866388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2532fe7-0779-408d-b40e-f130616bec57 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:26.867394 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2532fe7-0779-408d-b40e-f130616bec57 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:26.867867 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2532fe7-0779-408d-b40e-f130616bec57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:26.869977 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2532fe7-0779-408d-b40e-f130616bec57 None admin] RBAC: Adding query filter params (role.id=5456887a306649bf80a9ea7bd4282e97, scope.project.id=fcf4838b7b9e4303a80d7610c6bef277, user.id=27be4cdbaaec46c9b44d1f5474b281ae) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:26.870646 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2532fe7-0779-408d-b40e-f130616bec57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:26.873258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2532fe7-0779-408d-b40e-f130616bec57 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:26.883711 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 97/201] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:57:26 2022] GET /identity/v3/role_assignments?user.id=27be4cdbaaec46c9b44d1f5474b281ae&scope.project.id=fcf4838b7b9e4303a80d7610c6bef277&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 259 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:26.885289 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2532fe7-0779-408d-b40e-f130616bec57 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:26.901883 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:26.956860 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:27.000432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192149a-e4d7-4dd2-a3f0-5055abed754f None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:27.053430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2532fe7-0779-408d-b40e-f130616bec57 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:27.100525 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92227031-7a8f-4710-9b04-d5b79dc8a3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:27.102550 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92227031-7a8f-4710-9b04-d5b79dc8a3bd None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:27.103082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92227031-7a8f-4710-9b04-d5b79dc8a3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:27.103681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92227031-7a8f-4710-9b04-d5b79dc8a3bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:27.118109 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fae7e3-4847-4caf-9275-adc0f49a6bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:27.118808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fae7e3-4847-4caf-9275-adc0f49a6bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:27.119099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fae7e3-4847-4caf-9275-adc0f49a6bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:27.119380 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fae7e3-4847-4caf-9275-adc0f49a6bc2 None admin] PATH_INFO: `/v3/users/7699f51aaaca4af68243d0c6c54fc93d` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:27.120784 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fae7e3-4847-4caf-9275-adc0f49a6bc2 None admin] RBAC: Authorizing `identity:get_user(user_id=7699f51aaaca4af68243d0c6c54fc93d)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:27.122537 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fae7e3-4847-4caf-9275-adc0f49a6bc2 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:27.124244 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 105/202] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:26 2022] GET /identity/v3/users/7699f51aaaca4af68243d0c6c54fc93d => generated 289 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:27.130307 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fae7e3-4847-4caf-9275-adc0f49a6bc2 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:27.257270 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92227031-7a8f-4710-9b04-d5b79dc8a3bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:27.259448 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92227031-7a8f-4710-9b04-d5b79dc8a3bd None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:27.279038 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:27.420694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fae7e3-4847-4caf-9275-adc0f49a6bc2 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:27.469359 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 98/203] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:57:27.478043 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92227031-7a8f-4710-9b04-d5b79dc8a3bd None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:27.519465 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f202af-8b0d-4439-81d3-7e4f9fb50bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:27.520107 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f202af-8b0d-4439-81d3-7e4f9fb50bcf None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:27.520368 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f202af-8b0d-4439-81d3-7e4f9fb50bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:27.520681 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f202af-8b0d-4439-81d3-7e4f9fb50bcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:27.575579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:27.672669 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f202af-8b0d-4439-81d3-7e4f9fb50bcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:27.675696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f202af-8b0d-4439-81d3-7e4f9fb50bcf None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:27.699230 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92227031-7a8f-4710-9b04-d5b79dc8a3bd None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:27.732876 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:27.793851 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba1986-c3fb-442f-bc2a-2159dc0c6b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:27.794520 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba1986-c3fb-442f-bc2a-2159dc0c6b01 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:27.794804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba1986-c3fb-442f-bc2a-2159dc0c6b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:27.795088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba1986-c3fb-442f-bc2a-2159dc0c6b01 None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:27.796513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ba1986-c3fb-442f-bc2a-2159dc0c6b01 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4838b7b9e4303a80d7610c6bef277)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:27.798047 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ba1986-c3fb-442f-bc2a-2159dc0c6b01 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:27.799938 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 99/204] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:27 2022] GET /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277 => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:27.809645 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba1986-c3fb-442f-bc2a-2159dc0c6b01 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:27.960051 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 106/205] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 830 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:57:27.967915 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f202af-8b0d-4439-81d3-7e4f9fb50bcf None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:28.030174 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba1986-c3fb-442f-bc2a-2159dc0c6b01 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:28.079371 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:28.081355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297781e-88fd-4631-8301-eb6a7a230f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:28.081999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297781e-88fd-4631-8301-eb6a7a230f85 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:28.082523 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297781e-88fd-4631-8301-eb6a7a230f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:28.082523 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297781e-88fd-4631-8301-eb6a7a230f85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:28.083456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297781e-88fd-4631-8301-eb6a7a230f85 None admin] RBAC: Adding query filter params (role.id=e880d7a2b8c74bb1beb5015723a4e3d7, scope.project.id=fcf4838b7b9e4303a80d7610c6bef277, user.id=27be4cdbaaec46c9b44d1f5474b281ae) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:28.086187 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297781e-88fd-4631-8301-eb6a7a230f85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:28.093055 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297781e-88fd-4631-8301-eb6a7a230f85 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:28.100736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 100/206] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:57:27 2022] GET /identity/v3/role_assignments?user.id=27be4cdbaaec46c9b44d1f5474b281ae&scope.project.id=fcf4838b7b9e4303a80d7610c6bef277&role.id=e880d7a2b8c74bb1beb5015723a4e3d7 => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:28.151431 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f202af-8b0d-4439-81d3-7e4f9fb50bcf None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:28.192834 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e8081-1aed-4798-9b64-d0afa530dcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:28.193507 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e8081-1aed-4798-9b64-d0afa530dcab None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:28.193783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e8081-1aed-4798-9b64-d0afa530dcab None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:28.194056 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e8081-1aed-4798-9b64-d0afa530dcab None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:28.195517 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e8081-1aed-4798-9b64-d0afa530dcab None admin] RBAC: Authorizing `identity:get_project(project_id=cc8867738cf24c5db418fa4bd4cd2c01)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:28.197610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e8081-1aed-4798-9b64-d0afa530dcab None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:28.199773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 107/207] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:27 2022] GET /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:28.206360 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297781e-88fd-4631-8301-eb6a7a230f85 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:28.264398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:28.363310 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297781e-88fd-4631-8301-eb6a7a230f85 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:28.415746 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d389fc-8341-430b-b2e1-169ec6163cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:28.416929 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d389fc-8341-430b-b2e1-169ec6163cdd None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:28.417568 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d389fc-8341-430b-b2e1-169ec6163cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:28.418081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d389fc-8341-430b-b2e1-169ec6163cdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:28.419249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d389fc-8341-430b-b2e1-169ec6163cdd None admin] RBAC: Adding query filter params (role.id=46b7af1a00374c1989c960f2be43481e, scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01, user.id=7699f51aaaca4af68243d0c6c54fc93d) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:28.419766 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d389fc-8341-430b-b2e1-169ec6163cdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:28.421652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d389fc-8341-430b-b2e1-169ec6163cdd None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:28.429805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 101/208] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:57:28 2022] GET /identity/v3/role_assignments?user.id=7699f51aaaca4af68243d0c6c54fc93d&scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01&role.id=46b7af1a00374c1989c960f2be43481e => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:28.921130 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdfd39-9740-403d-b0f9-0593d697ca46 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:28.921527 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdfd39-9740-403d-b0f9-0593d697ca46 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:28.921725 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdfd39-9740-403d-b0f9-0593d697ca46 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:28.923995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 108/209] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:28.930360 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b793e5a-748f-4b2c-b94c-9a5a74dce3ad None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:28.930992 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b793e5a-748f-4b2c-b94c-9a5a74dce3ad None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:28.931573 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b793e5a-748f-4b2c-b94c-9a5a74dce3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:28.966393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-4b793e5a-748f-4b2c-b94c-9a5a74dce3ad None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:28.968328 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-4b793e5a-748f-4b2c-b94c-9a5a74dce3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:29.084885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5bc6e1-de92-4702-bd5c-5914bd4fb73a None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:29.086050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5bc6e1-de92-4702-bd5c-5914bd4fb73a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:29.086050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5bc6e1-de92-4702-bd5c-5914bd4fb73a None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:29.086192 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 102/210] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:29.086980 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 109/211] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:57:29.094483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f2213-63f7-48d6-a13c-91af066fa278 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:29.095929 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f2213-63f7-48d6-a13c-91af066fa278 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:29.095929 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f2213-63f7-48d6-a13c-91af066fa278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:29.114085 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5bc6e1-de92-4702-bd5c-5914bd4fb73a None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:29.126091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-842f2213-63f7-48d6-a13c-91af066fa278 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:29.128239 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-842f2213-63f7-48d6-a13c-91af066fa278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:29.170612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:29.221269 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 103/212] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:29.253031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5bc6e1-de92-4702-bd5c-5914bd4fb73a None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:29.263994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f2213-63f7-48d6-a13c-91af066fa278 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:29.277967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:29.293633 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772afec4-2bbf-477e-8981-60e5d0a87e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:29.294288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772afec4-2bbf-477e-8981-60e5d0a87e5f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:29.294570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772afec4-2bbf-477e-8981-60e5d0a87e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:29.294845 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772afec4-2bbf-477e-8981-60e5d0a87e5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:29.307514 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772afec4-2bbf-477e-8981-60e5d0a87e5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:29.313605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-772afec4-2bbf-477e-8981-60e5d0a87e5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:29.313605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:29.313605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:29.313605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:29.313605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:29.313605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:29.313605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:29.313605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:29.313605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:29.313605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:29.313605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:29.313605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:29.313605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:29.313605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:29.313605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:29.313605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:29.315265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:29.315265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:29.315265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:29.315265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:29.315265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:29.315265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:29.315265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:29.315265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:29.315265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:29.315265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:29.315265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:29.315265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:29.315265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:29.315265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:29.315265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:29.315265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:29.316818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:29.316818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:29.316818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:29.316818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:29.316818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:29.316818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:29.316818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:29.316818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:29.316818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:29.316818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:29.316818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:29.316818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:29.316818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 110/213] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:29.328881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772afec4-2bbf-477e-8981-60e5d0a87e5f None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:29.434529 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f2213-63f7-48d6-a13c-91af066fa278 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:29.491168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772afec4-2bbf-477e-8981-60e5d0a87e5f None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:29.492385 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388b1d4-7ced-4ea9-b185-e82cd8ac584c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:29.493460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388b1d4-7ced-4ea9-b185-e82cd8ac584c None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:29.493848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388b1d4-7ced-4ea9-b185-e82cd8ac584c None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:29.494335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388b1d4-7ced-4ea9-b185-e82cd8ac584c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:29.497232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388b1d4-7ced-4ea9-b185-e82cd8ac584c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:29.502882 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388b1d4-7ced-4ea9-b185-e82cd8ac584c None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:29.506325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 104/214] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:57:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:29.514464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388b1d4-7ced-4ea9-b185-e82cd8ac584c None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:29.544207 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc7ef9-8438-4a9b-9dd0-87cea6cbb4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:29.544957 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc7ef9-8438-4a9b-9dd0-87cea6cbb4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:29.545234 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc7ef9-8438-4a9b-9dd0-87cea6cbb4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:29.545507 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc7ef9-8438-4a9b-9dd0-87cea6cbb4a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:29.546219 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc7ef9-8438-4a9b-9dd0-87cea6cbb4a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:29.546516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc7ef9-8438-4a9b-9dd0-87cea6cbb4a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:29.548755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc7ef9-8438-4a9b-9dd0-87cea6cbb4a6 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:29.559914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:29.564568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 111/215] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:29.566757 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc7ef9-8438-4a9b-9dd0-87cea6cbb4a6 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:29.624344 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:29.653801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388b1d4-7ced-4ea9-b185-e82cd8ac584c None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:29.699795 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc7ef9-8438-4a9b-9dd0-87cea6cbb4a6 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:29.704580 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422b445-9269-4770-a6dd-ef1bc4649b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:29.705439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422b445-9269-4770-a6dd-ef1bc4649b7d None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:29.705870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422b445-9269-4770-a6dd-ef1bc4649b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:29.706293 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422b445-9269-4770-a6dd-ef1bc4649b7d None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:29.707143 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422b445-9269-4770-a6dd-ef1bc4649b7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:29.709435 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422b445-9269-4770-a6dd-ef1bc4649b7d None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:29.742501 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 105/216] 10.222.0.93 () {60 vars in 1132 bytes} [Mon Aug 1 14:57:29 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:57:29.750426 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4815441-e9a9-49de-9fa8-eb9cdcf6deba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:29.751854 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4815441-e9a9-49de-9fa8-eb9cdcf6deba None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:29.752602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4815441-e9a9-49de-9fa8-eb9cdcf6deba None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:29.753151 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4815441-e9a9-49de-9fa8-eb9cdcf6deba None admin] PATH_INFO: `/v3/users/c82fba8fe2cf46fc8b9f02e1f8605141` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:29.782698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4815441-e9a9-49de-9fa8-eb9cdcf6deba None admin] RBAC: Authorizing `identity:get_user(user_id=c82fba8fe2cf46fc8b9f02e1f8605141)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:29.784785 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4815441-e9a9-49de-9fa8-eb9cdcf6deba None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:29.787422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 112/217] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:29 2022] GET /identity/v3/users/c82fba8fe2cf46fc8b9f02e1f8605141 => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:29.795963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422b445-9269-4770-a6dd-ef1bc4649b7d None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:29.849370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832096d-1f49-43dc-b148-fa1605322fb7 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:29.849823 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832096d-1f49-43dc-b148-fa1605322fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:29.850242 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832096d-1f49-43dc-b148-fa1605322fb7 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:29.856896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 113/218] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:29 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:29.863119 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2e23c-f8e5-4a44-a8f6-c1a1a7e82d7f None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:29.863516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2e23c-f8e5-4a44-a8f6-c1a1a7e82d7f None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:29.863760 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2e23c-f8e5-4a44-a8f6-c1a1a7e82d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:29.889068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-4af2e23c-f8e5-4a44-a8f6-c1a1a7e82d7f None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:29.890402 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4af2e23c-f8e5-4a44-a8f6-c1a1a7e82d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:29.976151 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 114/219] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:57:29.984252 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:30.001762 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2e23c-f8e5-4a44-a8f6-c1a1a7e82d7f None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:30.008116 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422b445-9269-4770-a6dd-ef1bc4649b7d None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:30.048295 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:30.073225 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ae54b-4d5c-4755-a21a-353467938382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:30.074152 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ae54b-4d5c-4755-a21a-353467938382 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:30.074578 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ae54b-4d5c-4755-a21a-353467938382 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:30.075840 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ae54b-4d5c-4755-a21a-353467938382 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:30.190649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2e23c-f8e5-4a44-a8f6-c1a1a7e82d7f None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:30.200621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ae54b-4d5c-4755-a21a-353467938382 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:30.203506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ae54b-4d5c-4755-a21a-353467938382 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:30.242589 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40102a-1ab2-4916-90e9-59f3e2c7dcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:30.243252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40102a-1ab2-4916-90e9-59f3e2c7dcfe None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:30.243540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40102a-1ab2-4916-90e9-59f3e2c7dcfe None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:30.243807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40102a-1ab2-4916-90e9-59f3e2c7dcfe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:30.261684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40102a-1ab2-4916-90e9-59f3e2c7dcfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:30.266619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-0c40102a-1ab2-4916-90e9-59f3e2c7dcfe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:57:30.266619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:30.266619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:30.266619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:30.266619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:30.266619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:30.266619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:30.266619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:30.266619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:30.266619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:30.266619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:30.266619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:30.266619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:30.266619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:30.266619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:30.266619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:30.268504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:30.268504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:30.268504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:30.268504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:30.268504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:30.268504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:30.268504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:30.268504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:30.268504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:30.268504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:30.268504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:30.268504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:30.268504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:30.268504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:30.268504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:30.268504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:30.270090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:30.270090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:30.270090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:30.270090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:30.270090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:30.270090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:30.270090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:30.270090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:30.270090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:30.270090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:30.270090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:57:30.270090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:30.270090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 115/220] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:57:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:30.274836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40102a-1ab2-4916-90e9-59f3e2c7dcfe None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:30.400998 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:30.451367 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 106/221] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:57:30.457451 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ae54b-4d5c-4755-a21a-353467938382 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:30.477724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:30.506296 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40102a-1ab2-4916-90e9-59f3e2c7dcfe None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:30.574489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9685e31-dc91-4120-80dd-eea233007ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:30.576038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9685e31-dc91-4120-80dd-eea233007ead None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:30.576833 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9685e31-dc91-4120-80dd-eea233007ead None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:30.577587 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9685e31-dc91-4120-80dd-eea233007ead None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:30.579211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9685e31-dc91-4120-80dd-eea233007ead None admin] RBAC: Adding query filter params (name=admin) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:30.580103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9685e31-dc91-4120-80dd-eea233007ead None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:30.582982 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9685e31-dc91-4120-80dd-eea233007ead None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:30.601649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 116/222] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:30.612426 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9685e31-dc91-4120-80dd-eea233007ead None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:30.707979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ae54b-4d5c-4755-a21a-353467938382 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:30.792275 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac366e-5476-4257-981e-4ddab8d3456c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:30.793437 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac366e-5476-4257-981e-4ddab8d3456c None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:30.793943 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac366e-5476-4257-981e-4ddab8d3456c None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:30.794441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac366e-5476-4257-981e-4ddab8d3456c None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:30.797320 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac366e-5476-4257-981e-4ddab8d3456c None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4838b7b9e4303a80d7610c6bef277)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:30.800225 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac366e-5476-4257-981e-4ddab8d3456c None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:30.802534 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 107/223] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:30 2022] GET /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277 => generated 302 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:30.819135 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9685e31-dc91-4120-80dd-eea233007ead None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:30.824940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac366e-5476-4257-981e-4ddab8d3456c None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:30.891629 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81ed8d-e906-46bb-b73a-7d70d1b0fe08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:30.892748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81ed8d-e906-46bb-b73a-7d70d1b0fe08 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:30.893284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81ed8d-e906-46bb-b73a-7d70d1b0fe08 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:30.893657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81ed8d-e906-46bb-b73a-7d70d1b0fe08 None admin] PATH_INFO: `/v3/users/27be4cdbaaec46c9b44d1f5474b281ae` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:30.896956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81ed8d-e906-46bb-b73a-7d70d1b0fe08 None admin] RBAC: Authorizing `identity:get_user(user_id=27be4cdbaaec46c9b44d1f5474b281ae)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:30.898582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81ed8d-e906-46bb-b73a-7d70d1b0fe08 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:30.900429 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 117/224] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:30 2022] GET /identity/v3/users/27be4cdbaaec46c9b44d1f5474b281ae => generated 283 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:30.901877 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af9636-7c1c-4f2f-ba98-9a600591e9f9 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:30.902163 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af9636-7c1c-4f2f-ba98-9a600591e9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:30.902467 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af9636-7c1c-4f2f-ba98-9a600591e9f9 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:30.904083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 118/225] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:30.909128 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af9636-7c1c-4f2f-ba98-9a600591e9f9 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:30.924756 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:30.938920 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:30.990492 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac366e-5476-4257-981e-4ddab8d3456c None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:31.034529 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb1a42-2d14-4796-8d66-3231b1411f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:31.035927 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb1a42-2d14-4796-8d66-3231b1411f6c None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:31.036442 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb1a42-2d14-4796-8d66-3231b1411f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:31.036934 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb1a42-2d14-4796-8d66-3231b1411f6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:31.037906 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb1a42-2d14-4796-8d66-3231b1411f6c None admin] RBAC: Adding query filter params (role.id=c65b8ad9ade94760af715b3899bdd799, scope.project.id=fcf4838b7b9e4303a80d7610c6bef277, user.id=c82fba8fe2cf46fc8b9f02e1f8605141) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:31.038978 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb1a42-2d14-4796-8d66-3231b1411f6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:31.040704 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb1a42-2d14-4796-8d66-3231b1411f6c None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:31.055574 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 108/226] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:57:30 2022] GET /identity/v3/role_assignments?user.id=c82fba8fe2cf46fc8b9f02e1f8605141&scope.project.id=fcf4838b7b9e4303a80d7610c6bef277&role.id=c65b8ad9ade94760af715b3899bdd799 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:31.057490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce43862-a965-4376-9d4c-a13f906b0b79 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:31.057873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af9636-7c1c-4f2f-ba98-9a600591e9f9 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:31.058151 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce43862-a965-4376-9d4c-a13f906b0b79 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:31.058596 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce43862-a965-4376-9d4c-a13f906b0b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:31.088716 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-2ce43862-a965-4376-9d4c-a13f906b0b79 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:31.097265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-2ce43862-a965-4376-9d4c-a13f906b0b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:31.115923 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fd5b2-150a-4056-ae52-448da75ba366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:31.116672 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fd5b2-150a-4056-ae52-448da75ba366 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:31.117009 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fd5b2-150a-4056-ae52-448da75ba366 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:31.117317 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fd5b2-150a-4056-ae52-448da75ba366 None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:31.120441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fd5b2-150a-4056-ae52-448da75ba366 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4838b7b9e4303a80d7610c6bef277)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:31.122477 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fd5b2-150a-4056-ae52-448da75ba366 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:31.124368 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 119/227] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:30 2022] GET /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:31.130875 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fd5b2-150a-4056-ae52-448da75ba366 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:31.266337 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 109/228] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:31.305193 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce43862-a965-4376-9d4c-a13f906b0b79 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:31.343411 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:31.374617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fd5b2-150a-4056-ae52-448da75ba366 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:31.456334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:31.476373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1350d2c-2acc-4dc9-b92a-9fccbcf621a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:31.480039 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1350d2c-2acc-4dc9-b92a-9fccbcf621a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:31.480626 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1350d2c-2acc-4dc9-b92a-9fccbcf621a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:31.481109 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1350d2c-2acc-4dc9-b92a-9fccbcf621a1 None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277/users/27be4cdbaaec46c9b44d1f5474b281ae/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:31.486945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1350d2c-2acc-4dc9-b92a-9fccbcf621a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=fcf4838b7b9e4303a80d7610c6bef277, user_id=27be4cdbaaec46c9b44d1f5474b281ae, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:31.498045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1350d2c-2acc-4dc9-b92a-9fccbcf621a1 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:31.524160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 120/229] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:57:31 2022] PUT /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277/users/27be4cdbaaec46c9b44d1f5474b281ae/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:57:31.589554 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce43862-a965-4376-9d4c-a13f906b0b79 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:31.648937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef34c27-4a64-4dad-b804-62620eb6b72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:31.649517 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef34c27-4a64-4dad-b804-62620eb6b72b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:31.649798 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef34c27-4a64-4dad-b804-62620eb6b72b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:31.650070 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef34c27-4a64-4dad-b804-62620eb6b72b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:31.668268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef34c27-4a64-4dad-b804-62620eb6b72b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:31.673589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-3ef34c27-4a64-4dad-b804-62620eb6b72b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:31.673589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:31.673589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:31.673589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:31.673589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:31.673589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:31.673589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:31.673589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:31.673589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:31.673589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:31.673589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:31.673589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:31.673589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:31.673589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:31.673589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:31.673589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:31.675209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:31.675209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:31.675209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:31.675209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:31.675209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:31.675209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:31.675209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:31.675209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:31.675209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:31.675209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:31.675209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:31.675209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:31.675209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:31.675209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:31.675209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:31.675209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:31.676718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:31.676718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:31.676718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:31.676718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:31.676718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:31.676718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:31.676718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:31.676718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:31.676718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:31.676718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:31.676718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:31.676718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:31.676718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 110/230] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:57:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:57:31.684932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1350d2c-2acc-4dc9-b92a-9fccbcf621a1 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:31.914724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1350d2c-2acc-4dc9-b92a-9fccbcf621a1 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:31.978799 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68181c1-e988-434c-91a9-49548b742bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:31.979509 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68181c1-e988-434c-91a9-49548b742bcf None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:31.979884 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68181c1-e988-434c-91a9-49548b742bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:31.980249 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68181c1-e988-434c-91a9-49548b742bcf None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:31.981132 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68181c1-e988-434c-91a9-49548b742bcf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:31.981445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68181c1-e988-434c-91a9-49548b742bcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:31.982888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68181c1-e988-434c-91a9-49548b742bcf None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:31.993039 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 121/231] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:57:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:32.006057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef34c27-4a64-4dad-b804-62620eb6b72b None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:32.202781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef34c27-4a64-4dad-b804-62620eb6b72b None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:32.238030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:32.262356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd758e94-0a84-4209-99aa-008e8311f7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:32.263087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd758e94-0a84-4209-99aa-008e8311f7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:32.263365 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd758e94-0a84-4209-99aa-008e8311f7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:32.263660 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd758e94-0a84-4209-99aa-008e8311f7b0 None admin] PATH_INFO: `/v3/users/27be4cdbaaec46c9b44d1f5474b281ae` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:32.269596 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd758e94-0a84-4209-99aa-008e8311f7b0 None admin] RBAC: Authorizing `identity:get_user(user_id=27be4cdbaaec46c9b44d1f5474b281ae)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:32.271098 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd758e94-0a84-4209-99aa-008e8311f7b0 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:32.273929 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 111/232] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:32 2022] GET /identity/v3/users/27be4cdbaaec46c9b44d1f5474b281ae => generated 283 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:32.280041 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68181c1-e988-434c-91a9-49548b742bcf None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:32.292738 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:32.432487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68181c1-e988-434c-91a9-49548b742bcf None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:32.482686 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313bcaa-92f3-42f8-9a44-4706fdcc90a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:32.483255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313bcaa-92f3-42f8-9a44-4706fdcc90a2 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:32.483572 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313bcaa-92f3-42f8-9a44-4706fdcc90a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:32.483861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313bcaa-92f3-42f8-9a44-4706fdcc90a2 None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:32.485315 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9313bcaa-92f3-42f8-9a44-4706fdcc90a2 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4838b7b9e4303a80d7610c6bef277)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:32.486798 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9313bcaa-92f3-42f8-9a44-4706fdcc90a2 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:32.488615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 122/233] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:32 2022] GET /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:32.500042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd758e94-0a84-4209-99aa-008e8311f7b0 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:32.685181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd758e94-0a84-4209-99aa-008e8311f7b0 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:32.729710 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582744d-53c1-4fb2-a42f-8a5f399a0032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:32.730441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582744d-53c1-4fb2-a42f-8a5f399a0032 None admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:32.730788 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582744d-53c1-4fb2-a42f-8a5f399a0032 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:32.731070 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582744d-53c1-4fb2-a42f-8a5f399a0032 None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277/users/27be4cdbaaec46c9b44d1f5474b281ae/roles/e880d7a2b8c74bb1beb5015723a4e3d7` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:32.733425 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582744d-53c1-4fb2-a42f-8a5f399a0032 None admin] RBAC: Authorizing `identity:create_grant(project_id=fcf4838b7b9e4303a80d7610c6bef277, user_id=27be4cdbaaec46c9b44d1f5474b281ae, role_id=e880d7a2b8c74bb1beb5015723a4e3d7)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:32.734833 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582744d-53c1-4fb2-a42f-8a5f399a0032 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:32.758889 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 112/234] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:57:32 2022] PUT /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277/users/27be4cdbaaec46c9b44d1f5474b281ae/roles/e880d7a2b8c74bb1beb5015723a4e3d7 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:57:32.991380 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc50ca-9eef-46a7-be3d-6b6f723b6efa None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:32.992327 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc50ca-9eef-46a7-be3d-6b6f723b6efa None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:32.992516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc50ca-9eef-46a7-be3d-6b6f723b6efa None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:32.994347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 123/235] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:33.001286 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b0ec7-0eb2-41a5-ba1b-47dfeafa7421 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:33.001651 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b0ec7-0eb2-41a5-ba1b-47dfeafa7421 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:33.001878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b0ec7-0eb2-41a5-ba1b-47dfeafa7421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:33.025748 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-cf4b0ec7-0eb2-41a5-ba1b-47dfeafa7421 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:33.027015 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-cf4b0ec7-0eb2-41a5-ba1b-47dfeafa7421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:33.069340 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:33.128106 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 113/236] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:57:33.165705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc50ca-9eef-46a7-be3d-6b6f723b6efa None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:33.257568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:33.284554 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc50ca-9eef-46a7-be3d-6b6f723b6efa None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:33.324321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553eccf-852e-40a4-9b62-66784e3bda38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:33.325062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553eccf-852e-40a4-9b62-66784e3bda38 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:33.325348 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553eccf-852e-40a4-9b62-66784e3bda38 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:33.325625 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553eccf-852e-40a4-9b62-66784e3bda38 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:33.338200 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e553eccf-852e-40a4-9b62-66784e3bda38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:33.343929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-e553eccf-852e-40a4-9b62-66784e3bda38 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:57:33.343929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:33.343929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:33.343929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:33.343929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:33.343929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:33.343929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:33.343929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:33.343929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:33.343929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:33.343929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:33.343929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:33.343929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:33.343929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:33.343929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:33.343929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:33.344885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:33.344885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:33.344885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:33.344885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:33.344885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:33.344885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:33.344885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:33.344885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:33.344885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:33.344885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:33.344885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:33.344885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:33.344885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:33.344885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:33.344885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:33.344885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:33.345722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:33.345722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:33.345722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:33.345722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:33.345722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:33.345722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:33.345722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:33.345722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:33.345722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:33.345722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:33.345722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:57:33.345722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:33.345722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 124/237] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:33.352103 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b0ec7-0eb2-41a5-ba1b-47dfeafa7421 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:33.511734 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b0ec7-0eb2-41a5-ba1b-47dfeafa7421 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:33.570102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7795e-22ca-4108-9d65-b7f62fc602ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:33.570866 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7795e-22ca-4108-9d65-b7f62fc602ab None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:33.571198 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7795e-22ca-4108-9d65-b7f62fc602ab None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:33.571477 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7795e-22ca-4108-9d65-b7f62fc602ab None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:33.572362 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7795e-22ca-4108-9d65-b7f62fc602ab None admin] RBAC: Adding query filter params (name=reader) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:33.572696 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7795e-22ca-4108-9d65-b7f62fc602ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:33.574194 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7795e-22ca-4108-9d65-b7f62fc602ab None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:33.589113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 114/238] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:33.595416 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553eccf-852e-40a4-9b62-66784e3bda38 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:33.735671 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c4e2a-0232-4a03-bdbe-9d36f6b94695 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:33.736061 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c4e2a-0232-4a03-bdbe-9d36f6b94695 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:33.736288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c4e2a-0232-4a03-bdbe-9d36f6b94695 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:33.737893 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 115/239] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:57:33.739761 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553eccf-852e-40a4-9b62-66784e3bda38 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:33.743143 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af1e40-2870-4958-a616-5d06ff661452 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:33.743429 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af1e40-2870-4958-a616-5d06ff661452 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:33.743712 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af1e40-2870-4958-a616-5d06ff661452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:33.780488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-49af1e40-2870-4958-a616-5d06ff661452 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:33.782259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-49af1e40-2870-4958-a616-5d06ff661452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:33.838946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe353e-08ed-438c-bba8-0fa713794b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:33.839787 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe353e-08ed-438c-bba8-0fa713794b16 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:33.840205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe353e-08ed-438c-bba8-0fa713794b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:33.840382 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:33.840516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe353e-08ed-438c-bba8-0fa713794b16 None admin] PATH_INFO: `/v3/users/78bbc194a2314082a2a3051755bb1433` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:33.848642 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:33.872908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe353e-08ed-438c-bba8-0fa713794b16 None admin] RBAC: Authorizing `identity:get_user(user_id=78bbc194a2314082a2a3051755bb1433)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:33.876489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe353e-08ed-438c-bba8-0fa713794b16 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:33.879527 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 116/240] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:57:33.880747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 125/241] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:33 2022] GET /identity/v3/users/78bbc194a2314082a2a3051755bb1433 => generated 297 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:33.888120 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af1e40-2870-4958-a616-5d06ff661452 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:33.937293 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe353e-08ed-438c-bba8-0fa713794b16 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:34.081930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af1e40-2870-4958-a616-5d06ff661452 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:34.134908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe353e-08ed-438c-bba8-0fa713794b16 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:34.163438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537c934-f700-4891-ab57-782865a902e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:34.164321 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537c934-f700-4891-ab57-782865a902e8 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:34.164653 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537c934-f700-4891-ab57-782865a902e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:34.164941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537c934-f700-4891-ab57-782865a902e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:34.181928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909a281-a2be-46ee-9808-f680a31be0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:34.182929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909a281-a2be-46ee-9808-f680a31be0ec None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:34.183350 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909a281-a2be-46ee-9808-f680a31be0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:34.183764 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909a281-a2be-46ee-9808-f680a31be0ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:34.185181 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909a281-a2be-46ee-9808-f680a31be0ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:34.186830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909a281-a2be-46ee-9808-f680a31be0ec None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:34.188828 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 126/242] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:57:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:34.205984 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909a281-a2be-46ee-9808-f680a31be0ec None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:34.252983 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:34.298142 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:34.333480 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537c934-f700-4891-ab57-782865a902e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:34.336040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537c934-f700-4891-ab57-782865a902e8 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:34.377729 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909a281-a2be-46ee-9808-f680a31be0ec None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:34.443932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b7231-b06b-4c6c-88be-3bdf5fe1c859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:34.445655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b7231-b06b-4c6c-88be-3bdf5fe1c859 None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:34.446127 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b7231-b06b-4c6c-88be-3bdf5fe1c859 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:34.446613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b7231-b06b-4c6c-88be-3bdf5fe1c859 None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:34.447571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b7231-b06b-4c6c-88be-3bdf5fe1c859 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:34.449370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b7231-b06b-4c6c-88be-3bdf5fe1c859 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:34.478788 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 127/243] 10.222.0.93 () {60 vars in 1132 bytes} [Mon Aug 1 14:57:34 2022] POST /identity/v3/users => generated 293 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:57:34.480639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c34964-10cc-4e6f-ad3f-ab0a5d3cdcc3 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:34.481093 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c34964-10cc-4e6f-ad3f-ab0a5d3cdcc3 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:34.481617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c34964-10cc-4e6f-ad3f-ab0a5d3cdcc3 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:34.483705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 128/244] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:34.491997 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3cf65-127c-42cb-83f8-9126c526ba32 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:34.492584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3cf65-127c-42cb-83f8-9126c526ba32 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:34.493013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3cf65-127c-42cb-83f8-9126c526ba32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:34.529159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-69f3cf65-127c-42cb-83f8-9126c526ba32 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:34.538658 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-69f3cf65-127c-42cb-83f8-9126c526ba32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:34.578918 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 117/245] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:57:34.590133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537c934-f700-4891-ab57-782865a902e8 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:34.718449 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 129/246] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:57:34.749183 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3cf65-127c-42cb-83f8-9126c526ba32 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:34.750878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:34.822688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:34.848068 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537c934-f700-4891-ab57-782865a902e8 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:34.896911 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3cf65-127c-42cb-83f8-9126c526ba32 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:34.919317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dfea7-8894-470b-a14e-bb7b7985c13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:34.920884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dfea7-8894-470b-a14e-bb7b7985c13a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:34.921640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dfea7-8894-470b-a14e-bb7b7985c13a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:34.922430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dfea7-8894-470b-a14e-bb7b7985c13a None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:34.925039 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24dfea7-8894-470b-a14e-bb7b7985c13a None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4838b7b9e4303a80d7610c6bef277)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:34.928121 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24dfea7-8894-470b-a14e-bb7b7985c13a None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:34.932320 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 118/247] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:34 2022] GET /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277 => generated 302 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:34.941891 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dfea7-8894-470b-a14e-bb7b7985c13a None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:34.942169 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e83e32-37c5-404d-9273-c37f193e5f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:34.942789 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e83e32-37c5-404d-9273-c37f193e5f3b None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:34.943072 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e83e32-37c5-404d-9273-c37f193e5f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:34.943351 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e83e32-37c5-404d-9273-c37f193e5f3b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:34.957041 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e83e32-37c5-404d-9273-c37f193e5f3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:34.963510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-b5e83e32-37c5-404d-9273-c37f193e5f3b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:34.963510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:34.963510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:34.963510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:34.963510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:34.963510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:34.963510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:34.963510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:34.963510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:34.963510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:34.963510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:34.963510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:34.963510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:34.963510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:34.963510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:34.963510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:34.964548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:34.964548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:34.964548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:34.964548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:34.964548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:34.964548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:34.964548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:34.964548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:34.964548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:34.964548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:34.964548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:34.964548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:34.964548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:34.964548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:34.964548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:34.964548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:34.965602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:34.965602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:34.965602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:34.965602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:34.965602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:34.965602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:34.965602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:34.965602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:34.965602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:34.965602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:34.965602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:34.965602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:34.971535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 130/248] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:57:34.971784 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e83e32-37c5-404d-9273-c37f193e5f3b None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:35.098195 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e83e32-37c5-404d-9273-c37f193e5f3b None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:35.111608 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:35.144246 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd919d7-c14a-4b43-b052-f12584c34fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:35.144913 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd919d7-c14a-4b43-b052-f12584c34fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:35.145059 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dfea7-8894-470b-a14e-bb7b7985c13a None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:35.145255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd919d7-c14a-4b43-b052-f12584c34fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:35.145470 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd919d7-c14a-4b43-b052-f12584c34fb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:35.146189 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd919d7-c14a-4b43-b052-f12584c34fb7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:35.146499 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd919d7-c14a-4b43-b052-f12584c34fb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:35.148120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd919d7-c14a-4b43-b052-f12584c34fb7 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:35.162333 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 131/249] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:35.164856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd919d7-c14a-4b43-b052-f12584c34fb7 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:35.206004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:35.219143 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75c376-10f4-4042-b276-17c952b41e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:35.220625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75c376-10f4-4042-b276-17c952b41e51 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:35.221407 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75c376-10f4-4042-b276-17c952b41e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:35.222073 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75c376-10f4-4042-b276-17c952b41e51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:35.223722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75c376-10f4-4042-b276-17c952b41e51 None admin] RBAC: Adding query filter params (role.id=46b7af1a00374c1989c960f2be43481e, scope.project.id=fcf4838b7b9e4303a80d7610c6bef277, user.id=78bbc194a2314082a2a3051755bb1433) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:35.224510 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75c376-10f4-4042-b276-17c952b41e51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:35.227620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75c376-10f4-4042-b276-17c952b41e51 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:35.245187 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 119/250] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:57:34 2022] GET /identity/v3/role_assignments?user.id=78bbc194a2314082a2a3051755bb1433&scope.project.id=fcf4838b7b9e4303a80d7610c6bef277&role.id=46b7af1a00374c1989c960f2be43481e => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:35.327359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd919d7-c14a-4b43-b052-f12584c34fb7 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:35.380767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416df97e-4d0f-4f9b-89f6-bfea545641c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:35.381675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416df97e-4d0f-4f9b-89f6-bfea545641c7 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:35.382785 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416df97e-4d0f-4f9b-89f6-bfea545641c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:35.383137 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416df97e-4d0f-4f9b-89f6-bfea545641c7 None admin] PATH_INFO: `/v3/users/c82fba8fe2cf46fc8b9f02e1f8605141` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:35.385116 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416df97e-4d0f-4f9b-89f6-bfea545641c7 None admin] RBAC: Authorizing `identity:get_user(user_id=c82fba8fe2cf46fc8b9f02e1f8605141)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:35.386809 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416df97e-4d0f-4f9b-89f6-bfea545641c7 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:35.390680 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 132/251] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:35 2022] GET /identity/v3/users/c82fba8fe2cf46fc8b9f02e1f8605141 => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:35.397622 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75c376-10f4-4042-b276-17c952b41e51 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:35.406838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895fda7-96bc-4437-9640-7344691c7335 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:35.407825 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895fda7-96bc-4437-9640-7344691c7335 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:35.408709 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895fda7-96bc-4437-9640-7344691c7335 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:35.411312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 133/252] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:35.420765 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00225763-cfce-46a8-bad9-0675f9ee2765 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:35.421393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00225763-cfce-46a8-bad9-0675f9ee2765 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:35.421867 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00225763-cfce-46a8-bad9-0675f9ee2765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:35.441974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-00225763-cfce-46a8-bad9-0675f9ee2765 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:35.443892 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-00225763-cfce-46a8-bad9-0675f9ee2765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:35.484661 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:35.543772 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 134/253] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:57:35.568626 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00225763-cfce-46a8-bad9-0675f9ee2765 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:35.609638 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75c376-10f4-4042-b276-17c952b41e51 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:35.676449 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd7694-1781-4b6e-ac16-ade7e16d59ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:35.677493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd7694-1781-4b6e-ac16-ade7e16d59ff None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:35.677968 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd7694-1781-4b6e-ac16-ade7e16d59ff None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:35.678461 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd7694-1781-4b6e-ac16-ade7e16d59ff None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:35.681257 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd7694-1781-4b6e-ac16-ade7e16d59ff None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4838b7b9e4303a80d7610c6bef277)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:35.683989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd7694-1781-4b6e-ac16-ade7e16d59ff None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:35.685919 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00225763-cfce-46a8-bad9-0675f9ee2765 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:35.692667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 120/254] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:35 2022] GET /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277 => generated 302 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:35.698003 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd7694-1781-4b6e-ac16-ade7e16d59ff None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:35.723738 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebabef-0cb3-4e12-a901-b417cd0e6ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:35.724588 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebabef-0cb3-4e12-a901-b417cd0e6ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:35.724717 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebabef-0cb3-4e12-a901-b417cd0e6ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:35.725046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebabef-0cb3-4e12-a901-b417cd0e6ed4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:35.751659 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebabef-0cb3-4e12-a901-b417cd0e6ed4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:35.770176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-e0ebabef-0cb3-4e12-a901-b417cd0e6ed4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:57:35.770176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:35.770176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:35.770176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:35.770176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:35.770176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:35.770176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:35.770176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:35.770176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:35.770176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:35.770176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:35.770176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:35.770176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:35.770176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:35.770176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:35.770176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:35.771843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:35.771843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:35.771843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:35.771843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:35.771843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:35.771843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:35.771843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:35.771843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:35.771843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:35.771843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:35.771843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:35.771843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:35.771843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:35.771843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:35.771843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:35.771843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:35.773413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:35.773413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:35.773413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:35.773413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:35.773413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:35.773413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:35.773413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:35.773413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:35.773413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:35.773413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:35.773413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:57:35.773413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:35.777068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 135/255] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:57:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:35.781835 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebabef-0cb3-4e12-a901-b417cd0e6ed4 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:35.797700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:35.849827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd7694-1781-4b6e-ac16-ade7e16d59ff None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:35.850764 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:35.910343 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625804d3-d177-4397-830a-2df712a09a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:35.911021 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625804d3-d177-4397-830a-2df712a09a3b None admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:35.911302 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625804d3-d177-4397-830a-2df712a09a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:35.911579 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625804d3-d177-4397-830a-2df712a09a3b None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277/users/c82fba8fe2cf46fc8b9f02e1f8605141/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:35.918795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625804d3-d177-4397-830a-2df712a09a3b None admin] RBAC: Authorizing `identity:create_grant(project_id=fcf4838b7b9e4303a80d7610c6bef277, user_id=c82fba8fe2cf46fc8b9f02e1f8605141, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:35.921868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625804d3-d177-4397-830a-2df712a09a3b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:35.957053 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 121/256] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:57:35 2022] PUT /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277/users/c82fba8fe2cf46fc8b9f02e1f8605141/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:57:35.957053 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c44319-aa2d-4fc4-8d7e-e38b2decdfa3 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:35.957053 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c44319-aa2d-4fc4-8d7e-e38b2decdfa3 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:35.957053 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c44319-aa2d-4fc4-8d7e-e38b2decdfa3 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:35.957053 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 122/257] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:35.961126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24280b-35f5-4b19-b273-507759bd362e None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:35.961499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24280b-35f5-4b19-b273-507759bd362e None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:35.961667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24280b-35f5-4b19-b273-507759bd362e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:35.990982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-be24280b-35f5-4b19-b273-507759bd362e None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:35.996883 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-be24280b-35f5-4b19-b273-507759bd362e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:36.017510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebabef-0cb3-4e12-a901-b417cd0e6ed4 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:36.099493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b9144-7f01-4cd9-af5f-20472340adf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:36.100199 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b9144-7f01-4cd9-af5f-20472340adf7 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:36.100504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b9144-7f01-4cd9-af5f-20472340adf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:36.101353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b9144-7f01-4cd9-af5f-20472340adf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:36.102066 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b9144-7f01-4cd9-af5f-20472340adf7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:36.102370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b9144-7f01-4cd9-af5f-20472340adf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:36.103823 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b9144-7f01-4cd9-af5f-20472340adf7 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:36.122241 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 136/258] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:36.129333 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b9144-7f01-4cd9-af5f-20472340adf7 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:36.160816 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 123/259] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:57:36.221875 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24280b-35f5-4b19-b273-507759bd362e None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:36.333252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:36.356123 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b9144-7f01-4cd9-af5f-20472340adf7 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:36.374536 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:36.458806 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c72c4-7ee6-42ad-8a6a-99c256719085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:36.460678 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c72c4-7ee6-42ad-8a6a-99c256719085 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:36.461261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c72c4-7ee6-42ad-8a6a-99c256719085 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:36.461911 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c72c4-7ee6-42ad-8a6a-99c256719085 None admin] PATH_INFO: `/v3/users/27be4cdbaaec46c9b44d1f5474b281ae` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:36.472603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c72c4-7ee6-42ad-8a6a-99c256719085 None admin] RBAC: Authorizing `identity:get_user(user_id=27be4cdbaaec46c9b44d1f5474b281ae)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:36.472603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c72c4-7ee6-42ad-8a6a-99c256719085 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:36.474863 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 137/260] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:36 2022] GET /identity/v3/users/27be4cdbaaec46c9b44d1f5474b281ae => generated 283 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:36.488547 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c72c4-7ee6-42ad-8a6a-99c256719085 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:36.492690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24280b-35f5-4b19-b273-507759bd362e None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:36.577122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248d4d4-05ce-4f6b-aad4-f87c11cc7ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:36.577802 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248d4d4-05ce-4f6b-aad4-f87c11cc7ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:36.578091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248d4d4-05ce-4f6b-aad4-f87c11cc7ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:36.578373 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248d4d4-05ce-4f6b-aad4-f87c11cc7ba5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:36.613995 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9248d4d4-05ce-4f6b-aad4-f87c11cc7ba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:36.622868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-9248d4d4-05ce-4f6b-aad4-f87c11cc7ba5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:36.622868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:36.622868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:36.622868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:36.622868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:36.622868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:36.622868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:36.622868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:36.622868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:36.622868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:36.622868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:36.622868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:36.622868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:36.622868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:36.622868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:36.622868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:36.624224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:36.624224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:36.624224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:36.624224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:36.624224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:36.624224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:36.624224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:36.624224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:36.624224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:36.624224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:36.624224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:36.624224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:36.624224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:36.624224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:36.624224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:36.624224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:36.625183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:36.625183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:36.625183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:36.625183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:36.625183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:36.625183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:36.625183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:36.625183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:36.625183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:36.625183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:36.625183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:36.625183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:36.633635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 124/261] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:57:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:36.647038 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248d4d4-05ce-4f6b-aad4-f87c11cc7ba5 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:36.794749 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c72c4-7ee6-42ad-8a6a-99c256719085 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:36.866837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f03fc-c0a6-4714-9c6d-d2f738cb567e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:36.867719 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f03fc-c0a6-4714-9c6d-d2f738cb567e None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:36.868005 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f03fc-c0a6-4714-9c6d-d2f738cb567e None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:36.868275 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f03fc-c0a6-4714-9c6d-d2f738cb567e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:36.953853 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:36.971153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248d4d4-05ce-4f6b-aad4-f87c11cc7ba5 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:37.042046 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:37.136071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f03fc-c0a6-4714-9c6d-d2f738cb567e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:37.140123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e11514-82a6-4a35-bfa4-0b53d91aeaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:37.141675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e11514-82a6-4a35-bfa4-0b53d91aeaf9 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:37.142452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e11514-82a6-4a35-bfa4-0b53d91aeaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:37.143153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e11514-82a6-4a35-bfa4-0b53d91aeaf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:37.144804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e11514-82a6-4a35-bfa4-0b53d91aeaf9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:37.146401 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f03fc-c0a6-4714-9c6d-d2f738cb567e None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:37.147310 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e11514-82a6-4a35-bfa4-0b53d91aeaf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:37.151912 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e11514-82a6-4a35-bfa4-0b53d91aeaf9 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:37.186120 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 125/262] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:57:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 540 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:37.211563 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e11514-82a6-4a35-bfa4-0b53d91aeaf9 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:37.461393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 138/263] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 978 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:57:37.476160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f03fc-c0a6-4714-9c6d-d2f738cb567e None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:37.523276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e11514-82a6-4a35-bfa4-0b53d91aeaf9 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:37.634838 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46152af-4859-428a-a4c2-d1b781300b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:37.635964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46152af-4859-428a-a4c2-d1b781300b72 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:37.636384 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46152af-4859-428a-a4c2-d1b781300b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:37.636887 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46152af-4859-428a-a4c2-d1b781300b72 None admin] PATH_INFO: `/v3/users/27be4cdbaaec46c9b44d1f5474b281ae` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:37.640979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46152af-4859-428a-a4c2-d1b781300b72 None admin] RBAC: Authorizing `identity:get_user(user_id=27be4cdbaaec46c9b44d1f5474b281ae)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:37.644636 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46152af-4859-428a-a4c2-d1b781300b72 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:37.647992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:37.655519 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 126/264] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:37 2022] GET /identity/v3/users/27be4cdbaaec46c9b44d1f5474b281ae => generated 283 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:37.690802 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46152af-4859-428a-a4c2-d1b781300b72 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:37.724604 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f03fc-c0a6-4714-9c6d-d2f738cb567e None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:37.792060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04426f-11bb-4188-abb4-8b8ad235bf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:37.793493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04426f-11bb-4188-abb4-8b8ad235bf31 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:37.794214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04426f-11bb-4188-abb4-8b8ad235bf31 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:37.794885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04426f-11bb-4188-abb4-8b8ad235bf31 None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:37.797509 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04426f-11bb-4188-abb4-8b8ad235bf31 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4838b7b9e4303a80d7610c6bef277)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:37.799229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04426f-11bb-4188-abb4-8b8ad235bf31 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:37.806408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 139/265] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:37 2022] GET /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277 => generated 302 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:37.808078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04426f-11bb-4188-abb4-8b8ad235bf31 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:37.831776 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:37.929197 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04426f-11bb-4188-abb4-8b8ad235bf31 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:37.976203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:37.979591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46152af-4859-428a-a4c2-d1b781300b72 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:37.997586 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d3080-ae0c-42af-8e2f-ae57b5af47ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:37.998264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d3080-ae0c-42af-8e2f-ae57b5af47ea None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:37.998554 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d3080-ae0c-42af-8e2f-ae57b5af47ea None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:37.998832 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d3080-ae0c-42af-8e2f-ae57b5af47ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:37.999654 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620d3080-ae0c-42af-8e2f-ae57b5af47ea None admin] RBAC: Adding query filter params (role.id=5456887a306649bf80a9ea7bd4282e97, scope.project.id=fcf4838b7b9e4303a80d7610c6bef277, user.id=27be4cdbaaec46c9b44d1f5474b281ae) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:37.999968 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620d3080-ae0c-42af-8e2f-ae57b5af47ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:38.001576 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620d3080-ae0c-42af-8e2f-ae57b5af47ea None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:38.020967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 140/266] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:57:37 2022] GET /identity/v3/role_assignments?user.id=27be4cdbaaec46c9b44d1f5474b281ae&scope.project.id=fcf4838b7b9e4303a80d7610c6bef277&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:38.059052 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf192a-c7c0-4733-8cf5-a7715d1cdc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:38.060087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf192a-c7c0-4733-8cf5-a7715d1cdc00 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:38.060544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf192a-c7c0-4733-8cf5-a7715d1cdc00 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:38.061116 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf192a-c7c0-4733-8cf5-a7715d1cdc00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:38.196283 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf192a-c7c0-4733-8cf5-a7715d1cdc00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:38.199410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf192a-c7c0-4733-8cf5-a7715d1cdc00 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:38.299810 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:38.448603 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 127/267] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 759 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:57:38.457672 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d3080-ae0c-42af-8e2f-ae57b5af47ea None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:38.644961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d3080-ae0c-42af-8e2f-ae57b5af47ea None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:38.721948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab6e51-ae5c-4290-99d0-058e82bb12c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:38.723059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab6e51-ae5c-4290-99d0-058e82bb12c8 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:38.723637 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab6e51-ae5c-4290-99d0-058e82bb12c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:38.724148 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab6e51-ae5c-4290-99d0-058e82bb12c8 None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:38.729972 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab6e51-ae5c-4290-99d0-058e82bb12c8 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4838b7b9e4303a80d7610c6bef277)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:38.732973 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab6e51-ae5c-4290-99d0-058e82bb12c8 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:38.736761 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 141/268] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:38 2022] GET /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:38.743358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf192a-c7c0-4733-8cf5-a7715d1cdc00 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:38.883461 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c45ac-607b-404f-a2c8-df2600b9cf77 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:38.883982 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c45ac-607b-404f-a2c8-df2600b9cf77 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:38.884056 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c45ac-607b-404f-a2c8-df2600b9cf77 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:38.886458 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 142/269] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:38.892041 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95619933-5c15-4a38-ae86-0f720f516a7c None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:38.892349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95619933-5c15-4a38-ae86-0f720f516a7c None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:38.892736 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95619933-5c15-4a38-ae86-0f720f516a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:38.925154 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf192a-c7c0-4733-8cf5-a7715d1cdc00 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:38.946127 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-95619933-5c15-4a38-ae86-0f720f516a7c None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:38.948253 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-95619933-5c15-4a38-ae86-0f720f516a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:39.011944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0be3de-12f5-4642-b33a-39325ed3aacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:39.012594 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0be3de-12f5-4642-b33a-39325ed3aacf None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:39.012875 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0be3de-12f5-4642-b33a-39325ed3aacf None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:39.013144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0be3de-12f5-4642-b33a-39325ed3aacf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:39.013940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0be3de-12f5-4642-b33a-39325ed3aacf None admin] RBAC: Adding query filter params (role.id=e880d7a2b8c74bb1beb5015723a4e3d7, scope.project.id=fcf4838b7b9e4303a80d7610c6bef277, user.id=27be4cdbaaec46c9b44d1f5474b281ae) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:39.014245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0be3de-12f5-4642-b33a-39325ed3aacf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:39.015691 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0be3de-12f5-4642-b33a-39325ed3aacf None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:39.031607 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 128/270] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:57:38 2022] GET /identity/v3/role_assignments?user.id=27be4cdbaaec46c9b44d1f5474b281ae&scope.project.id=fcf4838b7b9e4303a80d7610c6bef277&role.id=e880d7a2b8c74bb1beb5015723a4e3d7 => generated 608 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:39.070278 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:39.094409 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 143/271] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:57:39.114936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca931f31-80d6-44e0-ac2a-bd50eb645fcc None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:39.115198 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca931f31-80d6-44e0-ac2a-bd50eb645fcc None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:39.115619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca931f31-80d6-44e0-ac2a-bd50eb645fcc None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:39.118159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 129/272] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:39.124500 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40536556-b701-4b84-9a4b-0ab661060e4a None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:39.124944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40536556-b701-4b84-9a4b-0ab661060e4a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:39.125099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40536556-b701-4b84-9a4b-0ab661060e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:39.136013 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca931f31-80d6-44e0-ac2a-bd50eb645fcc None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:39.153575 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-40536556-b701-4b84-9a4b-0ab661060e4a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:39.154863 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-40536556-b701-4b84-9a4b-0ab661060e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:39.157810 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:39.287073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 144/273] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:57:39.289444 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76b0d6-ab79-4eaa-ae9d-61e6587ea79d None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:39.289956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76b0d6-ab79-4eaa-ae9d-61e6587ea79d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:39.290429 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76b0d6-ab79-4eaa-ae9d-61e6587ea79d None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:39.292036 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 145/274] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:39.298254 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436569c-dad4-471a-b71f-fa40bf427a55 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:39.298813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436569c-dad4-471a-b71f-fa40bf427a55 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:39.299265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436569c-dad4-471a-b71f-fa40bf427a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:39.328571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-f436569c-dad4-471a-b71f-fa40bf427a55 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:39.331394 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f436569c-dad4-471a-b71f-fa40bf427a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:39.352027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca931f31-80d6-44e0-ac2a-bd50eb645fcc None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:39.401404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ce12f-da37-47c4-bc92-7f82b321abfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:39.402742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ce12f-da37-47c4-bc92-7f82b321abfb None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:39.403080 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ce12f-da37-47c4-bc92-7f82b321abfb None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:39.403442 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ce12f-da37-47c4-bc92-7f82b321abfb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:39.424359 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ce12f-da37-47c4-bc92-7f82b321abfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:39.435524 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 146/275] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:39.436340 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436569c-dad4-471a-b71f-fa40bf427a55 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:39.440907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-a39ce12f-da37-47c4-bc92-7f82b321abfb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:57:39.440907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:39.440907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:39.440907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:39.440907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:39.440907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:39.440907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:39.440907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:39.440907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:39.440907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:39.440907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:39.440907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:39.440907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:39.440907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:39.440907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:39.440907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:39.442686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:39.442686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:39.442686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:39.442686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:39.442686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:39.442686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:39.442686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:39.442686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:39.442686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:39.442686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:39.442686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:39.442686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:39.442686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:39.442686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:39.442686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:39.442686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:39.444953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:39.444953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:39.444953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:39.444953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:39.444953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:39.444953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:39.444953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:39.444953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:39.444953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:39.444953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:39.444953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:57:39.444953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:39.447286 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 130/276] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:57:39.449148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ce12f-da37-47c4-bc92-7f82b321abfb None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:39.520852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:39.548084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:39.593792 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ce12f-da37-47c4-bc92-7f82b321abfb None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:39.599406 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436569c-dad4-471a-b71f-fa40bf427a55 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:39.657954 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a54c923-45c3-4870-aaa0-8fde30e51282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:39.659035 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54c923-45c3-4870-aaa0-8fde30e51282 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:39.659224 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59116e09-0508-4c72-9be8-0ca6119937d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:39.659469 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54c923-45c3-4870-aaa0-8fde30e51282 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:39.659912 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54c923-45c3-4870-aaa0-8fde30e51282 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:39.660214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59116e09-0508-4c72-9be8-0ca6119937d3 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:39.660698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59116e09-0508-4c72-9be8-0ca6119937d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:39.661140 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59116e09-0508-4c72-9be8-0ca6119937d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:39.661233 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a54c923-45c3-4870-aaa0-8fde30e51282 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:39.661627 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a54c923-45c3-4870-aaa0-8fde30e51282 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:39.664196 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a54c923-45c3-4870-aaa0-8fde30e51282 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:39.678783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 131/277] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:39.681560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a54c923-45c3-4870-aaa0-8fde30e51282 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:39.685533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59116e09-0508-4c72-9be8-0ca6119937d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:39.696591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-59116e09-0508-4c72-9be8-0ca6119937d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:39.696591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:39.696591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:39.696591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:39.696591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:39.696591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:39.696591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:39.696591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:39.696591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:39.696591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:39.696591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:39.696591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:39.696591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:39.696591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:39.696591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:39.696591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:39.697574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:39.697574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:39.697574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:39.697574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:39.697574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:39.697574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:39.697574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:39.697574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:39.697574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:39.697574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:39.697574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:39.697574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:39.697574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:39.697574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:39.697574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:39.697574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:39.698455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:39.698455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:39.698455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:39.698455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:39.698455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:39.698455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:39.698455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:39.698455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:39.698455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:39.698455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:39.698455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:39.698455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:39.699062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 147/278] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:39.700519 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59116e09-0508-4c72-9be8-0ca6119937d3 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:39.840892 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a54c923-45c3-4870-aaa0-8fde30e51282 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:39.858936 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59116e09-0508-4c72-9be8-0ca6119937d3 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:39.874304 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:39.882962 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:39.904340 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb39a1-6fa4-4f1e-a4de-ddc909625958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:39.905320 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870af027-0e3b-41b8-b8b7-06bc1cc4fead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:39.905787 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb39a1-6fa4-4f1e-a4de-ddc909625958 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:39.905941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb39a1-6fa4-4f1e-a4de-ddc909625958 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:39.906036 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870af027-0e3b-41b8-b8b7-06bc1cc4fead None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:39.906249 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870af027-0e3b-41b8-b8b7-06bc1cc4fead None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:39.906418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb39a1-6fa4-4f1e-a4de-ddc909625958 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:39.906506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870af027-0e3b-41b8-b8b7-06bc1cc4fead None admin] PATH_INFO: `/v3/users/78bbc194a2314082a2a3051755bb1433` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:39.907684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870af027-0e3b-41b8-b8b7-06bc1cc4fead None admin] RBAC: Authorizing `identity:get_user(user_id=78bbc194a2314082a2a3051755bb1433)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:39.908396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb39a1-6fa4-4f1e-a4de-ddc909625958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:39.909189 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870af027-0e3b-41b8-b8b7-06bc1cc4fead None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:39.910822 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 148/279] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:39 2022] GET /identity/v3/users/78bbc194a2314082a2a3051755bb1433 => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:39.916083 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb39a1-6fa4-4f1e-a4de-ddc909625958 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:39.916268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870af027-0e3b-41b8-b8b7-06bc1cc4fead None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:39.916401 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 132/280] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:57:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:39.916401 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdac126-ead8-47c8-8e2d-1523d61f2050 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:39.916824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdac126-ead8-47c8-8e2d-1523d61f2050 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:39.917839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdac126-ead8-47c8-8e2d-1523d61f2050 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:39.919844 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 133/281] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:39.921288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdac126-ead8-47c8-8e2d-1523d61f2050 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:40.053256 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870af027-0e3b-41b8-b8b7-06bc1cc4fead None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:40.054095 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdac126-ead8-47c8-8e2d-1523d61f2050 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:40.091757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c2b74-1d3d-4543-968b-000ed68cae15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:40.092291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c2b74-1d3d-4543-968b-000ed68cae15 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:40.092599 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c2b74-1d3d-4543-968b-000ed68cae15 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:40.092883 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c2b74-1d3d-4543-968b-000ed68cae15 None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:40.094115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c2b74-1d3d-4543-968b-000ed68cae15 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4838b7b9e4303a80d7610c6bef277)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:40.095185 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25a27d-dc14-4095-ad03-73b800a35496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:40.095590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c2b74-1d3d-4543-968b-000ed68cae15 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:40.095918 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25a27d-dc14-4095-ad03-73b800a35496 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:40.096304 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25a27d-dc14-4095-ad03-73b800a35496 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:40.096660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25a27d-dc14-4095-ad03-73b800a35496 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:40.097348 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 134/282] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:39 2022] GET /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:40.098211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c2b74-1d3d-4543-968b-000ed68cae15 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:40.100814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25a27d-dc14-4095-ad03-73b800a35496 None admin] RBAC: Adding query filter params (name=member) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:40.101609 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25a27d-dc14-4095-ad03-73b800a35496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:40.104013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25a27d-dc14-4095-ad03-73b800a35496 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:40.114467 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 149/283] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:40.116631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932ee00-2115-4f61-8dc6-47ef01c64026 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:40.117066 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932ee00-2115-4f61-8dc6-47ef01c64026 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:40.117489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932ee00-2115-4f61-8dc6-47ef01c64026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:40.139931 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-7932ee00-2115-4f61-8dc6-47ef01c64026 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:40.141480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-7932ee00-2115-4f61-8dc6-47ef01c64026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:40.183905 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:40.195607 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:40.223529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 150/284] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:40.224439 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932ee00-2115-4f61-8dc6-47ef01c64026 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:40.253598 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c2b74-1d3d-4543-968b-000ed68cae15 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:40.294957 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d0d0d-91c1-49c8-a95a-8819ad5a7098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:40.295582 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d0d0d-91c1-49c8-a95a-8819ad5a7098 None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:40.295865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d0d0d-91c1-49c8-a95a-8819ad5a7098 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:40.296136 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d0d0d-91c1-49c8-a95a-8819ad5a7098 None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:40.296857 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d0d0d-91c1-49c8-a95a-8819ad5a7098 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:40.298335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d0d0d-91c1-49c8-a95a-8819ad5a7098 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:40.332539 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 135/285] 10.222.0.93 () {60 vars in 1132 bytes} [Mon Aug 1 14:57:40 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:57:40.333680 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d0d0d-91c1-49c8-a95a-8819ad5a7098 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:40.334836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932ee00-2115-4f61-8dc6-47ef01c64026 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:40.379258 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87659cc0-c951-4ae6-940c-fb595d732016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:40.380191 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87659cc0-c951-4ae6-940c-fb595d732016 None admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:40.380676 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87659cc0-c951-4ae6-940c-fb595d732016 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:40.381096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87659cc0-c951-4ae6-940c-fb595d732016 None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277/users/78bbc194a2314082a2a3051755bb1433/roles/46b7af1a00374c1989c960f2be43481e` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:40.384026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87659cc0-c951-4ae6-940c-fb595d732016 None admin] RBAC: Authorizing `identity:create_grant(project_id=fcf4838b7b9e4303a80d7610c6bef277, user_id=78bbc194a2314082a2a3051755bb1433, role_id=46b7af1a00374c1989c960f2be43481e)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:40.385761 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87659cc0-c951-4ae6-940c-fb595d732016 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:40.402474 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 151/286] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:57:40 2022] PUT /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277/users/78bbc194a2314082a2a3051755bb1433/roles/46b7af1a00374c1989c960f2be43481e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:57:40.403377 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87659cc0-c951-4ae6-940c-fb595d732016 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:40.454622 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:40.491167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d0d0d-91c1-49c8-a95a-8819ad5a7098 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:40.528010 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:40.552565 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7c332-6543-46b9-9358-99ef2b470b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:40.553359 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7c332-6543-46b9-9358-99ef2b470b95 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:40.554815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7c332-6543-46b9-9358-99ef2b470b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:40.554815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7c332-6543-46b9-9358-99ef2b470b95 None admin] PATH_INFO: `/v3/users/18eecdbf1aac4ae5b9739657790af051` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:40.561830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87659cc0-c951-4ae6-940c-fb595d732016 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:40.578071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7c332-6543-46b9-9358-99ef2b470b95 None admin] RBAC: Authorizing `identity:get_user(user_id=18eecdbf1aac4ae5b9739657790af051)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:40.579714 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7c332-6543-46b9-9358-99ef2b470b95 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:40.581796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 136/287] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:40 2022] GET /identity/v3/users/18eecdbf1aac4ae5b9739657790af051 => generated 293 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:40.588763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7c332-6543-46b9-9358-99ef2b470b95 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:40.614115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374543c8-e672-437a-be42-92310c541aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:40.614787 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374543c8-e672-437a-be42-92310c541aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:40.615068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374543c8-e672-437a-be42-92310c541aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:40.615339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374543c8-e672-437a-be42-92310c541aa1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:40.629346 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374543c8-e672-437a-be42-92310c541aa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:40.634469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-374543c8-e672-437a-be42-92310c541aa1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:40.634469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:40.634469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:40.634469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:40.634469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:40.634469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:40.634469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:40.634469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:40.634469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:40.634469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:40.634469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:40.634469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:40.634469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:40.634469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:40.634469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:40.634469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:40.635546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:40.635546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:40.635546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:40.635546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:40.635546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:40.635546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:40.635546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:40.635546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:40.635546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:40.635546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:40.635546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:40.635546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:40.635546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:40.635546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:40.635546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:40.635546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:40.636728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:40.636728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:40.636728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:40.636728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:40.636728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:40.636728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:40.636728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:40.636728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:40.636728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:40.636728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:40.636728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:40.636728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:40.636728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 152/288] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:40.643855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374543c8-e672-437a-be42-92310c541aa1 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:40.775310 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7c332-6543-46b9-9358-99ef2b470b95 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:40.808890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:40.837274 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374543c8-e672-437a-be42-92310c541aa1 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:40.845671 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee17bbd2-90d6-42ca-b4af-2f959174ed38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:40.847682 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17bbd2-90d6-42ca-b4af-2f959174ed38 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:40.848439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17bbd2-90d6-42ca-b4af-2f959174ed38 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:40.849256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17bbd2-90d6-42ca-b4af-2f959174ed38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:40.852761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee17bbd2-90d6-42ca-b4af-2f959174ed38 None admin] RBAC: Adding query filter params (role.id=c65b8ad9ade94760af715b3899bdd799, user.id=18eecdbf1aac4ae5b9739657790af051, scope.system=all) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:40.853721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee17bbd2-90d6-42ca-b4af-2f959174ed38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:40.856996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee17bbd2-90d6-42ca-b4af-2f959174ed38 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:40.865801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 137/289] 10.222.0.93 () {56 vars in 1308 bytes} [Mon Aug 1 14:57:40 2022] GET /identity/v3/role_assignments?user.id=18eecdbf1aac4ae5b9739657790af051&scope.system=all&role.id=c65b8ad9ade94760af715b3899bdd799 => generated 226 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:40.916081 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684a312-dbfa-46c3-9792-4e4b109986db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:40.918454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684a312-dbfa-46c3-9792-4e4b109986db None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:40.919321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684a312-dbfa-46c3-9792-4e4b109986db None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:40.920054 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684a312-dbfa-46c3-9792-4e4b109986db None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:40.921742 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684a312-dbfa-46c3-9792-4e4b109986db None admin] RBAC: Adding query filter params (name=member) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:40.922600 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684a312-dbfa-46c3-9792-4e4b109986db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:40.925548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684a312-dbfa-46c3-9792-4e4b109986db None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:40.943447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 153/290] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:40.952167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee17bbd2-90d6-42ca-b4af-2f959174ed38 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:41.036689 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:41.143113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee17bbd2-90d6-42ca-b4af-2f959174ed38 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:41.191754 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a021ed-cef8-4e92-b1d9-b0ef4b884f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:41.192386 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a021ed-cef8-4e92-b1d9-b0ef4b884f24 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:41.192686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a021ed-cef8-4e92-b1d9-b0ef4b884f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:41.192970 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a021ed-cef8-4e92-b1d9-b0ef4b884f24 None admin] PATH_INFO: `/v3/users/c82fba8fe2cf46fc8b9f02e1f8605141` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:41.194773 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a021ed-cef8-4e92-b1d9-b0ef4b884f24 None admin] RBAC: Authorizing `identity:get_user(user_id=c82fba8fe2cf46fc8b9f02e1f8605141)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:41.196310 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a021ed-cef8-4e92-b1d9-b0ef4b884f24 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:41.198561 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 138/291] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:40 2022] GET /identity/v3/users/c82fba8fe2cf46fc8b9f02e1f8605141 => generated 297 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:41.205389 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684a312-dbfa-46c3-9792-4e4b109986db None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:41.381518 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684a312-dbfa-46c3-9792-4e4b109986db None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:41.446578 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8237c-d509-4b92-bbf7-69f5e590d6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:41.447457 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8237c-d509-4b92-bbf7-69f5e590d6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:41.447926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8237c-d509-4b92-bbf7-69f5e590d6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:41.448392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8237c-d509-4b92-bbf7-69f5e590d6e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:41.487910 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:41.580504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8237c-d509-4b92-bbf7-69f5e590d6e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:41.582088 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8237c-d509-4b92-bbf7-69f5e590d6e7 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:41.727863 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 154/292] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:57:41.733134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a021ed-cef8-4e92-b1d9-b0ef4b884f24 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:41.910030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a021ed-cef8-4e92-b1d9-b0ef4b884f24 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:41.927162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:41.957830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306d091-1cba-4bb7-9198-1996490c6ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:41.958490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306d091-1cba-4bb7-9198-1996490c6ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:41.958774 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306d091-1cba-4bb7-9198-1996490c6ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:41.959052 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306d091-1cba-4bb7-9198-1996490c6ed3 None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:41.960320 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306d091-1cba-4bb7-9198-1996490c6ed3 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4838b7b9e4303a80d7610c6bef277)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:41.961837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306d091-1cba-4bb7-9198-1996490c6ed3 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:41.963556 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 139/293] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:41 2022] GET /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:41.975305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8237c-d509-4b92-bbf7-69f5e590d6e7 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:42.068195 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:42.155310 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8237c-d509-4b92-bbf7-69f5e590d6e7 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:42.212283 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76646dd8-1da3-45dc-b7ba-63f49f4e9914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:42.212939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76646dd8-1da3-45dc-b7ba-63f49f4e9914 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:42.213216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76646dd8-1da3-45dc-b7ba-63f49f4e9914 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:42.213486 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76646dd8-1da3-45dc-b7ba-63f49f4e9914 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:42.214272 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76646dd8-1da3-45dc-b7ba-63f49f4e9914 None admin] RBAC: Adding query filter params (role.id=c65b8ad9ade94760af715b3899bdd799, scope.project.id=fcf4838b7b9e4303a80d7610c6bef277, user.id=c82fba8fe2cf46fc8b9f02e1f8605141) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:42.214572 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76646dd8-1da3-45dc-b7ba-63f49f4e9914 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:42.216058 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76646dd8-1da3-45dc-b7ba-63f49f4e9914 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:42.225786 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 155/294] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:57:41 2022] GET /identity/v3/role_assignments?user.id=c82fba8fe2cf46fc8b9f02e1f8605141&scope.project.id=fcf4838b7b9e4303a80d7610c6bef277&role.id=c65b8ad9ade94760af715b3899bdd799 => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:43.397101 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bb076-44dc-48cb-b107-f58748cf7bf8 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:43.398478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bb076-44dc-48cb-b107-f58748cf7bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:43.398478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bb076-44dc-48cb-b107-f58748cf7bf8 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:43.399831 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 140/295] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:57:43.405162 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618f2b0-2df4-4ccd-9fb9-cc5c0989b484 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:43.406049 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618f2b0-2df4-4ccd-9fb9-cc5c0989b484 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:43.406049 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618f2b0-2df4-4ccd-9fb9-cc5c0989b484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:43.407431 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5f3eb-c075-4319-b053-a98ffe23a3bf None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:43.407709 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5f3eb-c075-4319-b053-a98ffe23a3bf None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:43.407963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5f3eb-c075-4319-b053-a98ffe23a3bf None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:43.410161 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 141/296] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:57:43.414217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508935b-e71e-46e4-aa6c-0ec14246e4b9 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:43.414514 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508935b-e71e-46e4-aa6c-0ec14246e4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:43.414770 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508935b-e71e-46e4-aa6c-0ec14246e4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:43.434275 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-b508935b-e71e-46e4-aa6c-0ec14246e4b9 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:43.434955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b508935b-e71e-46e4-aa6c-0ec14246e4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:43.445130 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-5618f2b0-2df4-4ccd-9fb9-cc5c0989b484 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:43.448903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5618f2b0-2df4-4ccd-9fb9-cc5c0989b484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:43.516714 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 142/297] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:57:43.535917 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b508935b-e71e-46e4-aa6c-0ec14246e4b9 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:43.600727 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 156/298] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:43.608071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f384e9-83bd-4eb3-8d30-96b22ced5dc4 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:43.608578 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f384e9-83bd-4eb3-8d30-96b22ced5dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:43.609293 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f384e9-83bd-4eb3-8d30-96b22ced5dc4 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:43.612030 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 157/299] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:43 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:43.621880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bc59d-5280-439a-8567-67c9abc27ac7 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:43.622656 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bc59d-5280-439a-8567-67c9abc27ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:43.623229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bc59d-5280-439a-8567-67c9abc27ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:43.645826 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-5b9bc59d-5280-439a-8567-67c9abc27ac7 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:43.647044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5b9bc59d-5280-439a-8567-67c9abc27ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:43.681939 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b508935b-e71e-46e4-aa6c-0ec14246e4b9 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:43.692294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:43.704693 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:43.739348 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c27e0-be83-4273-a532-2121352e695e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:43.740568 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c27e0-be83-4273-a532-2121352e695e None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:43.740868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c27e0-be83-4273-a532-2121352e695e None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:43.741725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c27e0-be83-4273-a532-2121352e695e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:43.754144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c27e0-be83-4273-a532-2121352e695e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:43.759739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-c51c27e0-be83-4273-a532-2121352e695e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:57:43.759739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:43.759739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:43.759739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:43.759739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:43.759739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:43.759739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:43.759739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:43.759739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:43.759739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:43.759739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:43.759739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:43.759739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:43.759739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:43.759739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:43.759739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:43.760896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:43.760896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:43.760896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:43.760896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:43.760896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:43.760896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:43.760896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:43.760896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:43.760896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:43.760896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:43.760896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:43.760896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:43.760896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:43.760896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:43.760896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:43.760896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:43.769276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:43.769276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:43.769276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:43.769276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:43.769276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:43.769276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:43.769276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:43.769276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:43.769276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:43.769276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:43.769276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:57:43.769276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:43.769276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 143/300] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:43.770030 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 158/301] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:43.770130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c27e0-be83-4273-a532-2121352e695e None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:43.770234 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bc59d-5280-439a-8567-67c9abc27ac7 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:43.887072 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bc59d-5280-439a-8567-67c9abc27ac7 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:43.890050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c27e0-be83-4273-a532-2121352e695e None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:43.926356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1c892-f860-4ea6-8358-8821e2271453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:43.927079 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1c892-f860-4ea6-8358-8821e2271453 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:43.927421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1c892-f860-4ea6-8358-8821e2271453 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:43.927760 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1c892-f860-4ea6-8358-8821e2271453 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:43.927860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4effff-e1df-4485-afe5-750da7771604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:43.929085 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1c892-f860-4ea6-8358-8821e2271453 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:43.929203 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4effff-e1df-4485-afe5-750da7771604 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:43.929469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1c892-f860-4ea6-8358-8821e2271453 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:43.929564 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4effff-e1df-4485-afe5-750da7771604 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:43.929889 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4effff-e1df-4485-afe5-750da7771604 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:43.930993 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1c892-f860-4ea6-8358-8821e2271453 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:43.931541 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4effff-e1df-4485-afe5-750da7771604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:43.933779 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4effff-e1df-4485-afe5-750da7771604 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:43.935516 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 144/302] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:57:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:43.936288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4effff-e1df-4485-afe5-750da7771604 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:43.943405 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 159/303] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:43.945321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1c892-f860-4ea6-8358-8821e2271453 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:43.979836 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:43.986823 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:44.058195 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4effff-e1df-4485-afe5-750da7771604 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:44.066624 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1c892-f860-4ea6-8358-8821e2271453 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:44.099471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af018d-cce6-4a34-be6c-7ea25f7c116b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:44.100118 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af018d-cce6-4a34-be6c-7ea25f7c116b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:44.100444 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af018d-cce6-4a34-be6c-7ea25f7c116b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:44.100842 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af018d-cce6-4a34-be6c-7ea25f7c116b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:44.104836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79dc4b-0912-4e53-9dd9-7298a51ed738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:44.105450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79dc4b-0912-4e53-9dd9-7298a51ed738 None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:44.105725 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79dc4b-0912-4e53-9dd9-7298a51ed738 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:44.105993 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79dc4b-0912-4e53-9dd9-7298a51ed738 None admin] PATH_INFO: `/v3/groups` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:44.106653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79dc4b-0912-4e53-9dd9-7298a51ed738 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:44.108081 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79dc4b-0912-4e53-9dd9-7298a51ed738 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:44.119107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af018d-cce6-4a34-be6c-7ea25f7c116b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:44.119107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-88af018d-cce6-4a34-be6c-7ea25f7c116b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:57:44.119107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:44.119107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:44.119107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:44.119107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:44.119107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:44.119107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:44.119107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:44.119107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:44.119107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:44.119107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:44.119107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:44.119107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:44.120144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:44.120144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:44.120144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:44.120144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:44.120144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:44.120144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:44.120144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:44.120144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:44.120144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:44.120144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:44.120144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:44.120144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:44.120144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:44.120144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:44.120144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:44.120144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:44.120144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:44.121201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:44.121201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:44.121201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:44.121201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:44.121201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:44.121201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:44.121201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:44.121201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:44.121201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:44.121201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:44.121201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:44.121201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:44.121201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:57:44.121201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:44.121201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 145/304] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:44.122529 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af018d-cce6-4a34-be6c-7ea25f7c116b None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:44.136127 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 160/305] 10.222.0.93 () {60 vars in 1134 bytes} [Mon Aug 1 14:57:43 2022] POST /identity/v3/groups => generated 229 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:57:44.138093 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855c9f6-85b4-46b0-a0b3-0ef317cdd146 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:44.138388 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855c9f6-85b4-46b0-a0b3-0ef317cdd146 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:44.138654 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855c9f6-85b4-46b0-a0b3-0ef317cdd146 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:44.140737 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 161/306] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:44.141708 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855c9f6-85b4-46b0-a0b3-0ef317cdd146 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:44.276547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:44.281781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:44.286702 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af018d-cce6-4a34-be6c-7ea25f7c116b None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:44.291251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855c9f6-85b4-46b0-a0b3-0ef317cdd146 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:44.332398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098cd989-1bd3-4400-b560-efdad4c387f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:44.333103 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cd989-1bd3-4400-b560-efdad4c387f0 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:44.333385 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cd989-1bd3-4400-b560-efdad4c387f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:44.333659 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cd989-1bd3-4400-b560-efdad4c387f0 None admin] PATH_INFO: `/v3/users/78bbc194a2314082a2a3051755bb1433` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:44.335155 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098cd989-1bd3-4400-b560-efdad4c387f0 None admin] RBAC: Authorizing `identity:get_user(user_id=78bbc194a2314082a2a3051755bb1433)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:44.336723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098cd989-1bd3-4400-b560-efdad4c387f0 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:44.338246 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8862b-6c06-4afb-86d0-060322668763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:44.338380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 146/307] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:44 2022] GET /identity/v3/users/78bbc194a2314082a2a3051755bb1433 => generated 297 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:44.338727 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8862b-6c06-4afb-86d0-060322668763 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:44.339003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8862b-6c06-4afb-86d0-060322668763 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:44.343347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8862b-6c06-4afb-86d0-060322668763 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:44.343517 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625a180-5806-42ee-b670-5c3a6e987267 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:44.343517 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625a180-5806-42ee-b670-5c3a6e987267 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:44.343517 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625a180-5806-42ee-b670-5c3a6e987267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:44.343732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8862b-6c06-4afb-86d0-060322668763 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:44.343732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8862b-6c06-4afb-86d0-060322668763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:44.343732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8862b-6c06-4afb-86d0-060322668763 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:44.355170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 162/308] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:44.356180 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8862b-6c06-4afb-86d0-060322668763 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:44.362359 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-d625a180-5806-42ee-b670-5c3a6e987267 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:44.363670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d625a180-5806-42ee-b670-5c3a6e987267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:44.447374 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 147/309] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:44.448278 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d625a180-5806-42ee-b670-5c3a6e987267 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:44.481138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8862b-6c06-4afb-86d0-060322668763 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:44.527978 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d34e1-e1ed-48e5-a4dc-221551f12e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:44.528879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d34e1-e1ed-48e5-a4dc-221551f12e61 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:44.529168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d34e1-e1ed-48e5-a4dc-221551f12e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:44.529723 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d34e1-e1ed-48e5-a4dc-221551f12e61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:44.562199 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:44.565291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:44.574865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d625a180-5806-42ee-b670-5c3a6e987267 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:44.609173 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d34e1-e1ed-48e5-a4dc-221551f12e61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:44.610746 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d34e1-e1ed-48e5-a4dc-221551f12e61 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:44.616390 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b1588-2d3b-4db4-bd2d-dd4390d86d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:44.617146 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b1588-2d3b-4db4-bd2d-dd4390d86d51 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:44.617508 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b1588-2d3b-4db4-bd2d-dd4390d86d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:44.617966 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b1588-2d3b-4db4-bd2d-dd4390d86d51 None admin] PATH_INFO: `/v3/users/edcb474286f847c19da8fa8ad2250ee0` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:44.634039 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b1588-2d3b-4db4-bd2d-dd4390d86d51 None admin] RBAC: Authorizing `identity:get_user(user_id=edcb474286f847c19da8fa8ad2250ee0)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:44.635517 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b1588-2d3b-4db4-bd2d-dd4390d86d51 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:44.642109 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 148/310] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:44 2022] GET /identity/v3/users/edcb474286f847c19da8fa8ad2250ee0 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:44.642109 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b1588-2d3b-4db4-bd2d-dd4390d86d51 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:44.735881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 163/311] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:57:44.737088 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d34e1-e1ed-48e5-a4dc-221551f12e61 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:44.755289 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b1588-2d3b-4db4-bd2d-dd4390d86d51 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:44.817857 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4cb231-a42b-4b98-90d1-6c819074c935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:44.818755 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4cb231-a42b-4b98-90d1-6c819074c935 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:44.819225 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4cb231-a42b-4b98-90d1-6c819074c935 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:44.819684 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4cb231-a42b-4b98-90d1-6c819074c935 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:44.838407 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4cb231-a42b-4b98-90d1-6c819074c935 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:44.838687 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:44.846720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-5d4cb231-a42b-4b98-90d1-6c819074c935 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:44.846720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:44.846720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:44.846720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:44.846720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:44.846720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:44.846720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:44.846720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:44.846720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:44.846720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:44.846720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:44.846720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:44.846720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:44.846720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:44.846720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:44.846720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:44.848775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:44.848775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:44.848775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:44.848775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:44.848775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:44.848775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:44.848775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:44.848775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:44.848775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:44.848775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:44.848775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:44.848775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:44.848775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:44.848775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:44.848775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:44.848775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:44.851104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:44.851104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:44.851104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:44.851104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:44.851104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:44.851104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:44.851104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:44.851104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:44.851104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:44.851104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:44.851104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:44.851104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:44.851104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 149/312] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:44.858256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4cb231-a42b-4b98-90d1-6c819074c935 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:44.883358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:44.930421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d34e1-e1ed-48e5-a4dc-221551f12e61 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:44.981055 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2f6a6-9b8e-49f5-a998-510e6834f275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:44.982107 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2f6a6-9b8e-49f5-a998-510e6834f275 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:44.982597 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2f6a6-9b8e-49f5-a998-510e6834f275 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:44.983091 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2f6a6-9b8e-49f5-a998-510e6834f275 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:44.984670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2f6a6-9b8e-49f5-a998-510e6834f275 None admin] RBAC: Adding query filter params (role.id=46b7af1a00374c1989c960f2be43481e, user.id=edcb474286f847c19da8fa8ad2250ee0, scope.system=all) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:44.985218 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2f6a6-9b8e-49f5-a998-510e6834f275 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:44.987924 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2f6a6-9b8e-49f5-a998-510e6834f275 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:44.998179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 164/313] 10.222.0.93 () {56 vars in 1308 bytes} [Mon Aug 1 14:57:44 2022] GET /identity/v3/role_assignments?user.id=edcb474286f847c19da8fa8ad2250ee0&scope.system=all&role.id=46b7af1a00374c1989c960f2be43481e => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:44.998824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2f6a6-9b8e-49f5-a998-510e6834f275 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:45.014581 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4cb231-a42b-4b98-90d1-6c819074c935 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:45.058436 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a6579-d825-400a-877e-48091e5dfd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:45.059571 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a6579-d825-400a-877e-48091e5dfd96 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:45.059860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a6579-d825-400a-877e-48091e5dfd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:45.060133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a6579-d825-400a-877e-48091e5dfd96 None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:45.061518 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a6579-d825-400a-877e-48091e5dfd96 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4838b7b9e4303a80d7610c6bef277)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:45.063016 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a6579-d825-400a-877e-48091e5dfd96 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:45.065084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 150/314] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:44 2022] GET /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:45.071755 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a6579-d825-400a-877e-48091e5dfd96 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:45.134643 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2f6a6-9b8e-49f5-a998-510e6834f275 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:45.165911 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:45.182870 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6166d-6dff-40f9-96d2-b3284bbf71a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:45.183906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6166d-6dff-40f9-96d2-b3284bbf71a0 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:45.184400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6166d-6dff-40f9-96d2-b3284bbf71a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:45.184926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6166d-6dff-40f9-96d2-b3284bbf71a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:45.186006 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6166d-6dff-40f9-96d2-b3284bbf71a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:45.186677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6166d-6dff-40f9-96d2-b3284bbf71a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:45.188371 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6166d-6dff-40f9-96d2-b3284bbf71a0 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:45.199291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 165/315] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:45.202142 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:45.207581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6166d-6dff-40f9-96d2-b3284bbf71a0 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:45.237842 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a6579-d825-400a-877e-48091e5dfd96 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:45.290295 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb352b2-69ab-4fc7-a22c-a498bb84602b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:45.291732 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb352b2-69ab-4fc7-a22c-a498bb84602b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:45.292326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb352b2-69ab-4fc7-a22c-a498bb84602b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:45.292796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb352b2-69ab-4fc7-a22c-a498bb84602b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:45.294251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb352b2-69ab-4fc7-a22c-a498bb84602b None admin] RBAC: Adding query filter params (role.id=46b7af1a00374c1989c960f2be43481e, scope.project.id=fcf4838b7b9e4303a80d7610c6bef277, user.id=78bbc194a2314082a2a3051755bb1433) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:45.295024 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb352b2-69ab-4fc7-a22c-a498bb84602b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:45.296745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb352b2-69ab-4fc7-a22c-a498bb84602b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:45.306124 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 151/316] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:57:45 2022] GET /identity/v3/role_assignments?user.id=78bbc194a2314082a2a3051755bb1433&scope.project.id=fcf4838b7b9e4303a80d7610c6bef277&role.id=46b7af1a00374c1989c960f2be43481e => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:45.343824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6166d-6dff-40f9-96d2-b3284bbf71a0 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:45.384328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b34f6-0c93-4dd0-a4a0-43610aa8f0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:45.385261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b34f6-0c93-4dd0-a4a0-43610aa8f0be None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:45.385682 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b34f6-0c93-4dd0-a4a0-43610aa8f0be None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:45.386118 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b34f6-0c93-4dd0-a4a0-43610aa8f0be None admin] PATH_INFO: `/v3/users/18eecdbf1aac4ae5b9739657790af051` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:45.387572 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b34f6-0c93-4dd0-a4a0-43610aa8f0be None admin] RBAC: Authorizing `identity:get_user(user_id=18eecdbf1aac4ae5b9739657790af051)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:45.389264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b34f6-0c93-4dd0-a4a0-43610aa8f0be None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:45.391438 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 166/317] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:45 2022] GET /identity/v3/users/18eecdbf1aac4ae5b9739657790af051 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:45.403768 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb352b2-69ab-4fc7-a22c-a498bb84602b None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:45.572816 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb352b2-69ab-4fc7-a22c-a498bb84602b None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:45.632065 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4959934-96c9-4aaa-89e6-7e8a70d1833b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:45.633453 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4959934-96c9-4aaa-89e6-7e8a70d1833b None admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:45.634112 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4959934-96c9-4aaa-89e6-7e8a70d1833b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:45.634764 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4959934-96c9-4aaa-89e6-7e8a70d1833b None admin] PATH_INFO: `/v3/system/users/18eecdbf1aac4ae5b9739657790af051/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:45.637424 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4959934-96c9-4aaa-89e6-7e8a70d1833b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=18eecdbf1aac4ae5b9739657790af051, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:45.639931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4959934-96c9-4aaa-89e6-7e8a70d1833b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:45.657828 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 152/318] 10.222.0.93 () {58 vars in 1334 bytes} [Mon Aug 1 14:57:45 2022] PUT /identity/v3/system/users/18eecdbf1aac4ae5b9739657790af051/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:57:46.509394 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c64060-248c-46a6-a0c9-a05d38ab7ab1 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:46.509912 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c64060-248c-46a6-a0c9-a05d38ab7ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:46.510138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c64060-248c-46a6-a0c9-a05d38ab7ab1 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:46.511689 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 167/319] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:46.517008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d2660-803f-4b28-976c-49d51a6d177c None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:46.517312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d2660-803f-4b28-976c-49d51a6d177c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:46.517561 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d2660-803f-4b28-976c-49d51a6d177c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:46.536812 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-ff7d2660-803f-4b28-976c-49d51a6d177c None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:46.538070 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ff7d2660-803f-4b28-976c-49d51a6d177c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:46.540205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:46.613065 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 153/320] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:46.637058 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c64060-248c-46a6-a0c9-a05d38ab7ab1 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:46.695813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:46.748355 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c64060-248c-46a6-a0c9-a05d38ab7ab1 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:46.788890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903ada9-faa0-480d-b965-fe7cfbf308ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:46.789537 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903ada9-faa0-480d-b965-fe7cfbf308ac None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:46.789818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903ada9-faa0-480d-b965-fe7cfbf308ac None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:46.790179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903ada9-faa0-480d-b965-fe7cfbf308ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:46.791358 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903ada9-faa0-480d-b965-fe7cfbf308ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:46.792879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903ada9-faa0-480d-b965-fe7cfbf308ac None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:46.794518 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 168/321] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:57:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:46.800546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d2660-803f-4b28-976c-49d51a6d177c None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:46.923381 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d2660-803f-4b28-976c-49d51a6d177c None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:46.962425 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f147f27-8154-4fba-bf87-4e1ff553ca58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:46.963045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f147f27-8154-4fba-bf87-4e1ff553ca58 None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:46.963325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f147f27-8154-4fba-bf87-4e1ff553ca58 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:46.963601 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f147f27-8154-4fba-bf87-4e1ff553ca58 None admin] PATH_INFO: `/v3/groups` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:46.964284 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f147f27-8154-4fba-bf87-4e1ff553ca58 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:46.965774 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f147f27-8154-4fba-bf87-4e1ff553ca58 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:46.979035 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:46.992237 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 154/322] 10.222.0.93 () {60 vars in 1134 bytes} [Mon Aug 1 14:57:46 2022] POST /identity/v3/groups => generated 226 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:57:47.335969 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cb5bc-d9cd-4954-9316-8ada626e3651 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:47.336488 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cb5bc-d9cd-4954-9316-8ada626e3651 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:47.337137 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cb5bc-d9cd-4954-9316-8ada626e3651 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:47.339747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 169/323] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:47.348679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14e345-9d55-47b5-bb96-eef59931f646 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:47.349273 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14e345-9d55-47b5-bb96-eef59931f646 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:47.349971 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14e345-9d55-47b5-bb96-eef59931f646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:47.375463 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-9e14e345-9d55-47b5-bb96-eef59931f646 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:47.376937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-9e14e345-9d55-47b5-bb96-eef59931f646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:47.459191 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 155/324] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:47.484227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cb5bc-d9cd-4954-9316-8ada626e3651 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:47.622933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cb5bc-d9cd-4954-9316-8ada626e3651 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:47.645113 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:47.661295 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebcd00-ee22-4bf9-bc74-0dae40a72bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:47.661991 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebcd00-ee22-4bf9-bc74-0dae40a72bcf None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:47.662263 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebcd00-ee22-4bf9-bc74-0dae40a72bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:47.662536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebcd00-ee22-4bf9-bc74-0dae40a72bcf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:47.674052 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebcd00-ee22-4bf9-bc74-0dae40a72bcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:47.680392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-e3ebcd00-ee22-4bf9-bc74-0dae40a72bcf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:57:47.680392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:47.680392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:47.680392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:47.680392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:47.680392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:47.680392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:47.680392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:47.680392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:47.680392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:47.680392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:47.680392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:47.680392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:47.680392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:47.680392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:47.680392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:47.681967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:47.681967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:47.681967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:47.681967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:47.681967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:47.681967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:47.681967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:47.681967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:47.681967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:47.681967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:47.681967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:47.681967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:47.681967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:47.681967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:47.681967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:47.681967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:47.683408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:47.683408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:47.683408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:47.683408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:47.683408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:47.683408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:47.683408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:47.683408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:47.683408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:47.683408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:47.683408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:57:47.683408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:47.683408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 170/325] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:57:47.693007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14e345-9d55-47b5-bb96-eef59931f646 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:47.820372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14e345-9d55-47b5-bb96-eef59931f646 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:47.856680 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:47.867613 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235b621-e63b-4bda-94fb-4112990872da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:47.868235 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235b621-e63b-4bda-94fb-4112990872da None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:47.868543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235b621-e63b-4bda-94fb-4112990872da None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:47.868810 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235b621-e63b-4bda-94fb-4112990872da None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:47.869532 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235b621-e63b-4bda-94fb-4112990872da None admin] RBAC: Adding query filter params (name=reader) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:47.869836 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235b621-e63b-4bda-94fb-4112990872da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:47.871236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235b621-e63b-4bda-94fb-4112990872da None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:47.883277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46de83f-5b02-4747-8651-8be6675947a7 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:47.883530 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46de83f-5b02-4747-8651-8be6675947a7 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:47.883899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46de83f-5b02-4747-8651-8be6675947a7 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:47.885391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 171/326] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:57:47.887122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 156/327] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:47.891292 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10825cf7-6f08-48fc-bed9-0b83c87c22a6 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:47.891604 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10825cf7-6f08-48fc-bed9-0b83c87c22a6 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:47.891857 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10825cf7-6f08-48fc-bed9-0b83c87c22a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:47.894550 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235b621-e63b-4bda-94fb-4112990872da None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:47.918276 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-10825cf7-6f08-48fc-bed9-0b83c87c22a6 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:47.919497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-10825cf7-6f08-48fc-bed9-0b83c87c22a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:48.078223 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 172/328] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:48.103296 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235b621-e63b-4bda-94fb-4112990872da None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:48.114275 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10825cf7-6f08-48fc-bed9-0b83c87c22a6 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:48.147036 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6494fe-c484-41d1-b753-52571a30078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:48.148050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6494fe-c484-41d1-b753-52571a30078f None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:48.149071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6494fe-c484-41d1-b753-52571a30078f None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:48.149575 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6494fe-c484-41d1-b753-52571a30078f None admin] PATH_INFO: `/v3/users/edcb474286f847c19da8fa8ad2250ee0` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:48.151149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6494fe-c484-41d1-b753-52571a30078f None admin] RBAC: Authorizing `identity:get_user(user_id=edcb474286f847c19da8fa8ad2250ee0)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:48.153234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6494fe-c484-41d1-b753-52571a30078f None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:48.155430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 157/329] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:47 2022] GET /identity/v3/users/edcb474286f847c19da8fa8ad2250ee0 => generated 293 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:48.161251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6494fe-c484-41d1-b753-52571a30078f None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:48.238327 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:48.299031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6494fe-c484-41d1-b753-52571a30078f None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:48.368001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2a570-061b-4865-9d33-d036f24c5484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:48.372267 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2a570-061b-4865-9d33-d036f24c5484 None admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:48.372887 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2a570-061b-4865-9d33-d036f24c5484 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:48.373414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2a570-061b-4865-9d33-d036f24c5484 None admin] PATH_INFO: `/v3/system/users/edcb474286f847c19da8fa8ad2250ee0/roles/46b7af1a00374c1989c960f2be43481e` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:48.376449 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2a570-061b-4865-9d33-d036f24c5484 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=edcb474286f847c19da8fa8ad2250ee0, role_id=46b7af1a00374c1989c960f2be43481e)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:48.381608 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2a570-061b-4865-9d33-d036f24c5484 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:48.398673 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 158/330] 10.222.0.93 () {58 vars in 1334 bytes} [Mon Aug 1 14:57:48 2022] PUT /identity/v3/system/users/edcb474286f847c19da8fa8ad2250ee0/roles/46b7af1a00374c1989c960f2be43481e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:57:48.403029 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10825cf7-6f08-48fc-bed9-0b83c87c22a6 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:48.418204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:48.446869 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80964064-f79b-4401-a194-1439d3c4d867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:48.447925 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80964064-f79b-4401-a194-1439d3c4d867 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:48.448435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80964064-f79b-4401-a194-1439d3c4d867 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:48.448996 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80964064-f79b-4401-a194-1439d3c4d867 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:48.464647 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80964064-f79b-4401-a194-1439d3c4d867 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:48.470028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-80964064-f79b-4401-a194-1439d3c4d867 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:48.470028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:48.470028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:48.470028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:48.470028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:48.470028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:48.470028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:48.470028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:48.470028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:48.470028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:48.470028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:48.470028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:48.470028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:48.470028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:48.470028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:48.470028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:48.470944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:48.470944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:48.470944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:48.470944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:48.470944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:48.470944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:48.470944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:48.470944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:48.470944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:48.470944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:48.470944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:48.470944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:48.470944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:48.470944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:48.470944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:48.470944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:48.471748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:48.471748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:48.471748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:48.471748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:48.471748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:48.471748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:48.471748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:48.471748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:48.471748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:48.471748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:48.471748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:48.471748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:48.474589 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 173/331] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:48.480684 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2a570-061b-4865-9d33-d036f24c5484 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:48.675728 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2a570-061b-4865-9d33-d036f24c5484 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:48.733025 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:48.757904 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e35a0a-d520-4565-98ce-3ae8dc5eb3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:48.759050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e35a0a-d520-4565-98ce-3ae8dc5eb3bc None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:48.759537 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e35a0a-d520-4565-98ce-3ae8dc5eb3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:48.760032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e35a0a-d520-4565-98ce-3ae8dc5eb3bc None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:48.761395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e35a0a-d520-4565-98ce-3ae8dc5eb3bc None admin] RBAC: Adding query filter params (name=member) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:48.761952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e35a0a-d520-4565-98ce-3ae8dc5eb3bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:48.764599 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e35a0a-d520-4565-98ce-3ae8dc5eb3bc None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:48.784859 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 159/332] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:48.791416 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80964064-f79b-4401-a194-1439d3c4d867 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:49.003320 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80964064-f79b-4401-a194-1439d3c4d867 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:49.070652 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ddc15-18a9-477c-9245-4435b07dfc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:49.071692 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ddc15-18a9-477c-9245-4435b07dfc2d None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:49.072231 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ddc15-18a9-477c-9245-4435b07dfc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:49.072705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ddc15-18a9-477c-9245-4435b07dfc2d None admin] PATH_INFO: `/v3/users/18eecdbf1aac4ae5b9739657790af051` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:49.074675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ddc15-18a9-477c-9245-4435b07dfc2d None admin] RBAC: Authorizing `identity:get_user(user_id=18eecdbf1aac4ae5b9739657790af051)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:49.077387 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ddc15-18a9-477c-9245-4435b07dfc2d None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:49.080400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 174/333] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:48 2022] GET /identity/v3/users/18eecdbf1aac4ae5b9739657790af051 => generated 293 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:49.089889 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e35a0a-d520-4565-98ce-3ae8dc5eb3bc None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:49.307256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e35a0a-d520-4565-98ce-3ae8dc5eb3bc None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:49.379619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b17b6-b8a2-40f4-8bb6-d93d94bbc5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:49.380736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b17b6-b8a2-40f4-8bb6-d93d94bbc5ca None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:49.381195 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b17b6-b8a2-40f4-8bb6-d93d94bbc5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:49.382145 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b17b6-b8a2-40f4-8bb6-d93d94bbc5ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:49.383615 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b17b6-b8a2-40f4-8bb6-d93d94bbc5ca None admin] RBAC: Adding query filter params (role.id=c65b8ad9ade94760af715b3899bdd799, user.id=18eecdbf1aac4ae5b9739657790af051, scope.system=all) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:49.384152 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b17b6-b8a2-40f4-8bb6-d93d94bbc5ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:49.386969 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b17b6-b8a2-40f4-8bb6-d93d94bbc5ca None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:49.417863 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 160/334] 10.222.0.93 () {56 vars in 1308 bytes} [Mon Aug 1 14:57:49 2022] GET /identity/v3/role_assignments?user.id=18eecdbf1aac4ae5b9739657790af051&scope.system=all&role.id=c65b8ad9ade94760af715b3899bdd799 => generated 510 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:50.580278 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5d125-a656-4193-ab86-fdaee94fe2bb None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:50.581364 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5d125-a656-4193-ab86-fdaee94fe2bb None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:50.581892 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5d125-a656-4193-ab86-fdaee94fe2bb None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:50.583482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 175/335] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:50.589367 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f06d87-07d0-40d2-a1ad-13620bbd1ee7 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:50.589857 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f06d87-07d0-40d2-a1ad-13620bbd1ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:50.589930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f06d87-07d0-40d2-a1ad-13620bbd1ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:50.614034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-87f06d87-07d0-40d2-a1ad-13620bbd1ee7 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:50.615311 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-87f06d87-07d0-40d2-a1ad-13620bbd1ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:50.667108 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:50.711342 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 161/336] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:50.764234 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5d125-a656-4193-ab86-fdaee94fe2bb None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:50.814802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:50.886668 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5d125-a656-4193-ab86-fdaee94fe2bb None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:50.934299 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813848d9-daad-40bc-96db-9c9e8e5198b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:50.934940 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813848d9-daad-40bc-96db-9c9e8e5198b0 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:50.935215 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813848d9-daad-40bc-96db-9c9e8e5198b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:50.935484 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813848d9-daad-40bc-96db-9c9e8e5198b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:50.950197 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813848d9-daad-40bc-96db-9c9e8e5198b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:50.955158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-813848d9-daad-40bc-96db-9c9e8e5198b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:50.955158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:50.955158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:50.955158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:50.955158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:50.955158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:50.955158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:50.955158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:50.955158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:50.955158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:50.955158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:50.955158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:50.955158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:50.955158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:50.955158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:50.955158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:50.956852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:50.956852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:50.956852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:50.956852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:50.956852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:50.956852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:50.956852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:50.956852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:50.956852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:50.956852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:50.956852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:50.956852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:50.956852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:50.956852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:50.956852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:50.956852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:50.958287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:50.958287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:50.958287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:50.958287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:50.958287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:50.958287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:50.958287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:50.958287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:50.958287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:50.958287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:50.958287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:50.958287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:50.958287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 176/337] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:50.962263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f06d87-07d0-40d2-a1ad-13620bbd1ee7 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:51.062712 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5e41a-0542-4c23-be66-572c17d4895d None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:51.063107 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5e41a-0542-4c23-be66-572c17d4895d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:51.063339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5e41a-0542-4c23-be66-572c17d4895d None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:51.064836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 177/338] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:51.072133 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6080c-54ec-4439-968f-568e899460f6 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:51.073083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6080c-54ec-4439-968f-568e899460f6 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:51.073372 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6080c-54ec-4439-968f-568e899460f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:51.110935 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-a5c6080c-54ec-4439-968f-568e899460f6 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:51.114832 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a5c6080c-54ec-4439-968f-568e899460f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:51.153522 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f06d87-07d0-40d2-a1ad-13620bbd1ee7 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:51.243220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9320b-7ef6-4c60-adc9-60b1f2495759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:51.243853 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9320b-7ef6-4c60-adc9-60b1f2495759 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:51.244136 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9320b-7ef6-4c60-adc9-60b1f2495759 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:51.244411 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9320b-7ef6-4c60-adc9-60b1f2495759 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:51.245619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9320b-7ef6-4c60-adc9-60b1f2495759 None admin] RBAC: Adding query filter params (name=member) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:51.245619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9320b-7ef6-4c60-adc9-60b1f2495759 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:51.247152 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9320b-7ef6-4c60-adc9-60b1f2495759 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:51.264896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 162/339] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:51.268193 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 178/340] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:57:51.272525 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9320b-7ef6-4c60-adc9-60b1f2495759 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:51.313810 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6080c-54ec-4439-968f-568e899460f6 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:51.324903 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:51.375517 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:51.414251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9320b-7ef6-4c60-adc9-60b1f2495759 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:51.501679 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6080c-54ec-4439-968f-568e899460f6 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:51.513855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1aea8-d3b3-4616-af32-eed9be70a7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:51.514789 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1aea8-d3b3-4616-af32-eed9be70a7dc None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:51.515211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1aea8-d3b3-4616-af32-eed9be70a7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:51.515626 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1aea8-d3b3-4616-af32-eed9be70a7dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:51.561228 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba18a34-20a2-45d6-9351-3a23945f1792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:51.561997 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba18a34-20a2-45d6-9351-3a23945f1792 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:51.562473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba18a34-20a2-45d6-9351-3a23945f1792 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:51.562894 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba18a34-20a2-45d6-9351-3a23945f1792 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:51.582705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba18a34-20a2-45d6-9351-3a23945f1792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:51.590660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-3ba18a34-20a2-45d6-9351-3a23945f1792 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:51.590660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:51.590660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:51.590660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:51.590660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:51.590660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:51.590660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:51.590660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:51.590660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:51.590660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:51.590660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:51.590660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:51.590660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:51.590660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:51.590660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:51.590660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:51.591807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:51.591807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:51.591807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:51.591807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:51.591807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:51.591807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:51.591807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:51.591807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:51.591807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:51.591807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:51.591807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:51.591807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:51.591807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:51.591807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:51.591807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:51.591807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:51.592778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:51.592778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:51.592778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:51.592778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:51.592778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:51.592778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:51.592778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:51.592778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:51.592778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:51.592778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:51.592778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:51.592778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:51.595131 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 179/341] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:57:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:57:51.601102 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba18a34-20a2-45d6-9351-3a23945f1792 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:51.606795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1aea8-d3b3-4616-af32-eed9be70a7dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:51.613565 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1aea8-d3b3-4616-af32-eed9be70a7dc None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:51.705418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:51.789949 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 163/342] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:57:51.791585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db000f-5009-44a3-906f-4e15e055019a None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:51.792027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db000f-5009-44a3-906f-4e15e055019a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:51.792449 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db000f-5009-44a3-906f-4e15e055019a None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:51.794075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 164/343] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:51.799678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7d85d-c312-4257-b38f-b954ac8b3787 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:51.800311 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7d85d-c312-4257-b38f-b954ac8b3787 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:51.800819 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7d85d-c312-4257-b38f-b954ac8b3787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:51.803388 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:51.822732 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-27a7d85d-c312-4257-b38f-b954ac8b3787 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:51.822916 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba18a34-20a2-45d6-9351-3a23945f1792 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:51.824335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-27a7d85d-c312-4257-b38f-b954ac8b3787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:51.890901 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5992729-0ce2-4207-95ca-d2cc3017d9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:51.893038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5992729-0ce2-4207-95ca-d2cc3017d9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:51.895440 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5992729-0ce2-4207-95ca-d2cc3017d9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:51.895922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5992729-0ce2-4207-95ca-d2cc3017d9e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:51.896930 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5992729-0ce2-4207-95ca-d2cc3017d9e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:51.899006 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5992729-0ce2-4207-95ca-d2cc3017d9e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:51.900626 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5992729-0ce2-4207-95ca-d2cc3017d9e9 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:51.920237 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 180/344] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:57:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:51.923417 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5992729-0ce2-4207-95ca-d2cc3017d9e9 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:51.943584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 165/345] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:57:51.945627 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7d85d-c312-4257-b38f-b954ac8b3787 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:52.082605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:52.138958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5992729-0ce2-4207-95ca-d2cc3017d9e9 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:52.183952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb06fc-839e-421f-b5c9-5140a1226030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:52.184977 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb06fc-839e-421f-b5c9-5140a1226030 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:52.185463 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb06fc-839e-421f-b5c9-5140a1226030 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:52.185971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb06fc-839e-421f-b5c9-5140a1226030 None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:52.186415 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7d85d-c312-4257-b38f-b954ac8b3787 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:52.187586 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb06fc-839e-421f-b5c9-5140a1226030 None admin] RBAC: Authorizing `identity:get_project(project_id=cc8867738cf24c5db418fa4bd4cd2c01)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:52.190498 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb06fc-839e-421f-b5c9-5140a1226030 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:52.192524 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 181/346] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:51 2022] GET /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:52.193548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb06fc-839e-421f-b5c9-5140a1226030 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:52.220005 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:52.275456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b324f73-dd86-4f8e-8d42-3f0b90a517a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:52.276680 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b324f73-dd86-4f8e-8d42-3f0b90a517a4 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:52.277204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b324f73-dd86-4f8e-8d42-3f0b90a517a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:52.277663 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b324f73-dd86-4f8e-8d42-3f0b90a517a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:52.366389 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b324f73-dd86-4f8e-8d42-3f0b90a517a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:52.368373 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b324f73-dd86-4f8e-8d42-3f0b90a517a4 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:52.410414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb06fc-839e-421f-b5c9-5140a1226030 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:52.448980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:52.464175 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf3a77-1008-49a1-ad7e-349908adc734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:52.464899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf3a77-1008-49a1-ad7e-349908adc734 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:52.465178 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf3a77-1008-49a1-ad7e-349908adc734 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:52.465560 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf3a77-1008-49a1-ad7e-349908adc734 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:52.482322 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf3a77-1008-49a1-ad7e-349908adc734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:52.492734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-adaf3a77-1008-49a1-ad7e-349908adc734 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:52.492734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:52.492734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:52.492734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:52.492734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:52.492734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:52.492734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:52.492734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:52.492734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:52.492734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:52.492734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:52.492734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:52.492734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:52.492734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:52.492734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:52.492734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:52.493763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:52.493763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:52.493763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:52.493763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:52.493763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:52.493763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:52.493763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:52.493763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:52.493763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:52.493763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:52.493763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:52.493763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:52.493763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:52.493763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:52.493763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:52.493763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:52.495129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:52.495129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:52.495129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:52.495129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:52.495129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:52.495129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:52.495129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:52.495129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:52.495129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:52.495129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:52.495129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:52.495129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:52.495129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 182/347] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:52.496125 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf3a77-1008-49a1-ad7e-349908adc734 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:52.497061 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 166/348] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:57:52.503547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b324f73-dd86-4f8e-8d42-3f0b90a517a4 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:52.621780 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b324f73-dd86-4f8e-8d42-3f0b90a517a4 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:52.627622 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:52.663434 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33c30f-49c7-4d01-a03a-8aff4bf816e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:52.663862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf3a77-1008-49a1-ad7e-349908adc734 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:52.664725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33c30f-49c7-4d01-a03a-8aff4bf816e0 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:52.665343 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33c30f-49c7-4d01-a03a-8aff4bf816e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:52.665861 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33c30f-49c7-4d01-a03a-8aff4bf816e0 None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:52.667371 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33c30f-49c7-4d01-a03a-8aff4bf816e0 None admin] RBAC: Authorizing `identity:get_project(project_id=cc8867738cf24c5db418fa4bd4cd2c01)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:52.669785 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33c30f-49c7-4d01-a03a-8aff4bf816e0 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:52.672347 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 167/349] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:52 2022] GET /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:52.673687 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33c30f-49c7-4d01-a03a-8aff4bf816e0 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:52.719939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236ca1a-3e7f-4a31-b3e4-6b8fb42a29c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:52.720544 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236ca1a-3e7f-4a31-b3e4-6b8fb42a29c2 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:52.720827 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236ca1a-3e7f-4a31-b3e4-6b8fb42a29c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:52.721101 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236ca1a-3e7f-4a31-b3e4-6b8fb42a29c2 None admin] PATH_INFO: `/v3/groups/56a3c992a6384e7f8444cc92b4180f64` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:52.729000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:52.734156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236ca1a-3e7f-4a31-b3e4-6b8fb42a29c2 None admin] RBAC: Authorizing `identity:get_group(group_id=56a3c992a6384e7f8444cc92b4180f64)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:52.735638 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236ca1a-3e7f-4a31-b3e4-6b8fb42a29c2 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:52.737806 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 183/350] 10.222.0.93 () {56 vars in 1180 bytes} [Mon Aug 1 14:57:52 2022] GET /identity/v3/groups/56a3c992a6384e7f8444cc92b4180f64 => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:52.738571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236ca1a-3e7f-4a31-b3e4-6b8fb42a29c2 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:52.798882 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33c30f-49c7-4d01-a03a-8aff4bf816e0 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:52.838318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a05cff-9686-4fa0-8db6-8dc3c3d44382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:52.839597 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a05cff-9686-4fa0-8db6-8dc3c3d44382 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:52.840101 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a05cff-9686-4fa0-8db6-8dc3c3d44382 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:52.841844 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a05cff-9686-4fa0-8db6-8dc3c3d44382 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:52.842930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a05cff-9686-4fa0-8db6-8dc3c3d44382 None admin] RBAC: Adding query filter params (name=member) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:52.843461 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a05cff-9686-4fa0-8db6-8dc3c3d44382 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:52.845244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a05cff-9686-4fa0-8db6-8dc3c3d44382 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:52.855770 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 168/351] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:52.857047 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a05cff-9686-4fa0-8db6-8dc3c3d44382 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:52.893615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236ca1a-3e7f-4a31-b3e4-6b8fb42a29c2 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:52.932056 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d1ee8-020e-428d-9792-d7a45053465b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:52.932750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d1ee8-020e-428d-9792-d7a45053465b None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:52.933026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d1ee8-020e-428d-9792-d7a45053465b None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:52.933297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d1ee8-020e-428d-9792-d7a45053465b None admin] PATH_INFO: `/v3/groups/56a3c992a6384e7f8444cc92b4180f64` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:52.934456 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d1ee8-020e-428d-9792-d7a45053465b None admin] RBAC: Authorizing `identity:get_group(group_id=56a3c992a6384e7f8444cc92b4180f64)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:52.935934 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d1ee8-020e-428d-9792-d7a45053465b None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:52.940961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 184/352] 10.222.0.93 () {56 vars in 1180 bytes} [Mon Aug 1 14:57:52 2022] GET /identity/v3/groups/56a3c992a6384e7f8444cc92b4180f64 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:52.941973 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5ba61-e1ef-4ef8-9c8e-f966b3b98482 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:52.944846 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5ba61-e1ef-4ef8-9c8e-f966b3b98482 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:52.945130 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5ba61-e1ef-4ef8-9c8e-f966b3b98482 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:52.946637 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 185/353] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:52.947244 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5ba61-e1ef-4ef8-9c8e-f966b3b98482 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:52.966509 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:52.973269 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:52.985758 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a05cff-9686-4fa0-8db6-8dc3c3d44382 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:53.030886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5be13-0246-4c11-be5f-6b48a0ad7866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:53.031514 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5be13-0246-4c11-be5f-6b48a0ad7866 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:53.031790 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5be13-0246-4c11-be5f-6b48a0ad7866 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:53.032061 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5be13-0246-4c11-be5f-6b48a0ad7866 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:53.032981 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5be13-0246-4c11-be5f-6b48a0ad7866 None admin] RBAC: Adding query filter params (group.id=56a3c992a6384e7f8444cc92b4180f64, role.id=c65b8ad9ade94760af715b3899bdd799, scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:53.033287 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5be13-0246-4c11-be5f-6b48a0ad7866 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:53.036606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5be13-0246-4c11-be5f-6b48a0ad7866 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:53.042957 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 169/354] 10.222.0.93 () {56 vars in 1376 bytes} [Mon Aug 1 14:57:52 2022] GET /identity/v3/role_assignments?group.id=56a3c992a6384e7f8444cc92b4180f64&scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01&role.id=c65b8ad9ade94760af715b3899bdd799 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:53.044047 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96015817-402e-42c5-83b6-92047c25cddd None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:53.044327 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96015817-402e-42c5-83b6-92047c25cddd None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:53.044643 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96015817-402e-42c5-83b6-92047c25cddd None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:53.046105 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 170/355] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:53.047093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96015817-402e-42c5-83b6-92047c25cddd None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:53.094804 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5ba61-e1ef-4ef8-9c8e-f966b3b98482 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:53.149761 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd5292-abaf-40d1-95f0-c45a0d685586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:53.150429 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd5292-abaf-40d1-95f0-c45a0d685586 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:53.150921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd5292-abaf-40d1-95f0-c45a0d685586 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:53.151197 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd5292-abaf-40d1-95f0-c45a0d685586 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:53.215779 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96015817-402e-42c5-83b6-92047c25cddd None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:53.260219 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd5292-abaf-40d1-95f0-c45a0d685586 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:53.261236 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd5292-abaf-40d1-95f0-c45a0d685586 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:53.267340 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9da5d8-8f3b-408f-bfd4-832c2ac2e6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:53.267968 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9da5d8-8f3b-408f-bfd4-832c2ac2e6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:53.268246 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9da5d8-8f3b-408f-bfd4-832c2ac2e6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:53.268555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9da5d8-8f3b-408f-bfd4-832c2ac2e6f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:53.269347 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9da5d8-8f3b-408f-bfd4-832c2ac2e6f0 None admin] RBAC: Adding query filter params (group.id=56a3c992a6384e7f8444cc92b4180f64, role.id=e880d7a2b8c74bb1beb5015723a4e3d7, scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:53.269648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9da5d8-8f3b-408f-bfd4-832c2ac2e6f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:53.271098 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9da5d8-8f3b-408f-bfd4-832c2ac2e6f0 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:53.278772 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:53.279739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 171/356] 10.222.0.93 () {56 vars in 1376 bytes} [Mon Aug 1 14:57:53 2022] GET /identity/v3/role_assignments?group.id=56a3c992a6384e7f8444cc92b4180f64&scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01&role.id=e880d7a2b8c74bb1beb5015723a4e3d7 => generated 260 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:53.280995 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade33174-4a65-44c5-9f93-cdbd92401435 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:53.281135 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade33174-4a65-44c5-9f93-cdbd92401435 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:53.281351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade33174-4a65-44c5-9f93-cdbd92401435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:53.307917 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-ade33174-4a65-44c5-9f93-cdbd92401435 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:53.307917 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ade33174-4a65-44c5-9f93-cdbd92401435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:53.332855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:53.421219 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 172/357] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:53.423363 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15600ab-e3c5-44f4-bd50-6536415b35ed None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:53.423744 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15600ab-e3c5-44f4-bd50-6536415b35ed None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:53.424232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15600ab-e3c5-44f4-bd50-6536415b35ed None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:53.425818 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 173/358] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:53.428427 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d81be8-1ea2-46d7-95a0-1b57c76f7c15 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:53.429593 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d81be8-1ea2-46d7-95a0-1b57c76f7c15 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:53.429963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d81be8-1ea2-46d7-95a0-1b57c76f7c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:53.436537 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 186/359] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:57:53.438253 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47207fcc-3dc0-4a48-b481-fcce1da14669 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:53.438750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47207fcc-3dc0-4a48-b481-fcce1da14669 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:53.439318 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47207fcc-3dc0-4a48-b481-fcce1da14669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:53.450188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-35d81be8-1ea2-46d7-95a0-1b57c76f7c15 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:53.451421 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-35d81be8-1ea2-46d7-95a0-1b57c76f7c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:53.472211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-47207fcc-3dc0-4a48-b481-fcce1da14669 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:53.473824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-47207fcc-3dc0-4a48-b481-fcce1da14669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:53.552858 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 174/360] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:53.554801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d81be8-1ea2-46d7-95a0-1b57c76f7c15 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:53.577945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 187/361] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:53.579150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47207fcc-3dc0-4a48-b481-fcce1da14669 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:53.640260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:53.683932 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:53.706864 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d81be8-1ea2-46d7-95a0-1b57c76f7c15 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:53.738808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47207fcc-3dc0-4a48-b481-fcce1da14669 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:53.755990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e6287-91a7-4f6a-a7de-716922a4eb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:53.757398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e6287-91a7-4f6a-a7de-716922a4eb13 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:53.757676 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e6287-91a7-4f6a-a7de-716922a4eb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:53.757949 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e6287-91a7-4f6a-a7de-716922a4eb13 None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:53.759291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e6287-91a7-4f6a-a7de-716922a4eb13 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4838b7b9e4303a80d7610c6bef277)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:53.760909 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e6287-91a7-4f6a-a7de-716922a4eb13 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:53.762999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 175/362] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:53 2022] GET /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:53.764033 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e6287-91a7-4f6a-a7de-716922a4eb13 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:53.788381 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae22d9-4f10-4245-94e8-227315ee3ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:53.789078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae22d9-4f10-4245-94e8-227315ee3ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:53.789354 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae22d9-4f10-4245-94e8-227315ee3ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:53.789627 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae22d9-4f10-4245-94e8-227315ee3ba3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:53.807027 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae22d9-4f10-4245-94e8-227315ee3ba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:53.812618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-0bae22d9-4f10-4245-94e8-227315ee3ba3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:53.812618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:53.812618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:53.812618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:53.812618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:53.812618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:53.812618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:53.812618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:53.812618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:53.812618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:53.812618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:53.812618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:53.812618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:53.812618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:53.812618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:53.812618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:53.813594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:53.813594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:53.813594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:53.813594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:53.813594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:53.813594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:53.813594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:53.813594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:53.813594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:53.813594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:53.813594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:53.813594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:53.813594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:53.813594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:53.813594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:53.813594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:53.814435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:53.814435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:53.814435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:53.814435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:53.814435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:53.814435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:53.814435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:53.814435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:53.814435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:53.814435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:53.814435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:53.814435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:53.814435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 188/363] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:57:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:53.817268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae22d9-4f10-4245-94e8-227315ee3ba3 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:53.912350 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e6287-91a7-4f6a-a7de-716922a4eb13 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:53.962719 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d608840-1e0c-4cfd-8d69-e5e98a9d5240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:53.963388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d608840-1e0c-4cfd-8d69-e5e98a9d5240 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:53.963670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d608840-1e0c-4cfd-8d69-e5e98a9d5240 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:53.963948 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d608840-1e0c-4cfd-8d69-e5e98a9d5240 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:53.977485 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d608840-1e0c-4cfd-8d69-e5e98a9d5240 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:53.985667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-6d608840-1e0c-4cfd-8d69-e5e98a9d5240 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:57:53.985667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:53.985667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:53.985667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:53.985667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:53.985667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:53.985667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:53.985667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:53.985667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:53.985667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:53.985667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:53.985667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:53.985667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:53.985667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:53.985667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:53.985667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:53.986621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:53.986621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:53.986621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:53.986621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:53.986621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:53.986621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:53.986621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:53.986621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:53.986621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:53.986621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:53.986621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:53.986621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:53.986621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:53.986621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:53.986621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:53.986621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:53.987418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:53.987418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:53.987418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:53.987418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:53.987418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:53.987418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:53.987418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:53.987418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:53.987418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:53.987418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:53.987418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:57:53.987418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:53.992599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:53.995124 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 176/364] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:53.995124 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d608840-1e0c-4cfd-8d69-e5e98a9d5240 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:54.006929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae22d9-4f10-4245-94e8-227315ee3ba3 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:54.048428 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:54.088425 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13821ae3-839c-4913-984c-5e81759c5774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:54.089412 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13821ae3-839c-4913-984c-5e81759c5774 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:54.090683 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13821ae3-839c-4913-984c-5e81759c5774 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:54.091176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13821ae3-839c-4913-984c-5e81759c5774 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:54.106780 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13821ae3-839c-4913-984c-5e81759c5774 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:54.113156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-13821ae3-839c-4913-984c-5e81759c5774 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:57:54.113156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:54.113156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:54.113156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:54.113156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:54.113156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:54.113156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:54.113156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:54.113156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:54.113156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:54.113156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:54.113156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:54.113156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:54.113156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:54.113156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:54.113156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:54.114136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:54.114136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:54.114136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:54.114136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:54.114136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:54.114136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:54.114136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:54.114136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:54.114136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:54.114136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:54.114136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:54.114136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:54.114136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:54.114136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:54.114136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:54.114136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:54.114942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:54.114942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:54.114942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:54.114942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:54.114942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:54.114942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:54.114942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:54.114942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:54.114942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:54.114942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:54.114942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:57:54.114942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:54.117404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 189/365] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:57:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:54.118519 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13821ae3-839c-4913-984c-5e81759c5774 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:54.200176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d608840-1e0c-4cfd-8d69-e5e98a9d5240 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:54.262644 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e1cf9-4217-4ad5-ad04-1acb5131481a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:54.263767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e1cf9-4217-4ad5-ad04-1acb5131481a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:54.264299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e1cf9-4217-4ad5-ad04-1acb5131481a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:54.264854 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e1cf9-4217-4ad5-ad04-1acb5131481a None admin] PATH_INFO: `/v3/groups/56a3c992a6384e7f8444cc92b4180f64` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:54.267063 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e1cf9-4217-4ad5-ad04-1acb5131481a None admin] RBAC: Authorizing `identity:get_group(group_id=56a3c992a6384e7f8444cc92b4180f64)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:54.269477 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e1cf9-4217-4ad5-ad04-1acb5131481a None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:54.272131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 177/366] 10.222.0.93 () {56 vars in 1180 bytes} [Mon Aug 1 14:57:53 2022] GET /identity/v3/groups/56a3c992a6384e7f8444cc92b4180f64 => generated 226 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:54.273567 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e1cf9-4217-4ad5-ad04-1acb5131481a None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:54.309593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13821ae3-839c-4913-984c-5e81759c5774 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:54.375589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:54.380087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0712e-775d-42c8-9a59-2bc6866d4f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:54.382349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0712e-775d-42c8-9a59-2bc6866d4f5b None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:54.382349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0712e-775d-42c8-9a59-2bc6866d4f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:54.382768 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0712e-775d-42c8-9a59-2bc6866d4f5b None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:54.384014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0712e-775d-42c8-9a59-2bc6866d4f5b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:54.384522 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0712e-775d-42c8-9a59-2bc6866d4f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:54.387207 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0712e-775d-42c8-9a59-2bc6866d4f5b None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:54.403410 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 190/367] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:57:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:54.404939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0712e-775d-42c8-9a59-2bc6866d4f5b None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:54.405176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e1cf9-4217-4ad5-ad04-1acb5131481a None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:54.449390 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee87e4-6713-4590-bdd1-e11978125749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:54.450599 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee87e4-6713-4590-bdd1-e11978125749 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:54.451108 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee87e4-6713-4590-bdd1-e11978125749 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:54.451582 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee87e4-6713-4590-bdd1-e11978125749 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:54.452526 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee87e4-6713-4590-bdd1-e11978125749 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:54.453050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee87e4-6713-4590-bdd1-e11978125749 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:54.454659 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee87e4-6713-4590-bdd1-e11978125749 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:54.456656 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:54.467496 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 178/368] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:54.468826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee87e4-6713-4590-bdd1-e11978125749 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:54.602779 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0712e-775d-42c8-9a59-2bc6866d4f5b None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:54.605378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee87e4-6713-4590-bdd1-e11978125749 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:54.645833 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c5feb-c458-46c6-84cb-3667f9216ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:54.646949 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c5feb-c458-46c6-84cb-3667f9216ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:54.647458 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c5feb-c458-46c6-84cb-3667f9216ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:54.647956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c5feb-c458-46c6-84cb-3667f9216ad7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:54.649042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c5feb-c458-46c6-84cb-3667f9216ad7 None admin] RBAC: Adding query filter params (group.id=56a3c992a6384e7f8444cc92b4180f64, role.id=c65b8ad9ade94760af715b3899bdd799, scope.project.id=fcf4838b7b9e4303a80d7610c6bef277) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:54.649682 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c5feb-c458-46c6-84cb-3667f9216ad7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:54.651619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c5feb-c458-46c6-84cb-3667f9216ad7 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:54.658526 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 179/369] 10.222.0.93 () {56 vars in 1376 bytes} [Mon Aug 1 14:57:54 2022] GET /identity/v3/role_assignments?group.id=56a3c992a6384e7f8444cc92b4180f64&scope.project.id=fcf4838b7b9e4303a80d7610c6bef277&role.id=c65b8ad9ade94760af715b3899bdd799 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:54.659837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c5feb-c458-46c6-84cb-3667f9216ad7 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:54.667111 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:54.678000 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a6b3d-5b6d-41f5-a2d3-90f37849f3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:54.679590 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a6b3d-5b6d-41f5-a2d3-90f37849f3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:54.680107 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a6b3d-5b6d-41f5-a2d3-90f37849f3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:54.680813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a6b3d-5b6d-41f5-a2d3-90f37849f3d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:54.682403 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a6b3d-5b6d-41f5-a2d3-90f37849f3d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:54.682736 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a6b3d-5b6d-41f5-a2d3-90f37849f3d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:54.685600 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a6b3d-5b6d-41f5-a2d3-90f37849f3d0 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:54.704159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 191/370] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:54.705779 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a6b3d-5b6d-41f5-a2d3-90f37849f3d0 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:54.798974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c5feb-c458-46c6-84cb-3667f9216ad7 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:54.841069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760f3b8-391b-4bf5-8c66-7d7fe3a58177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:54.842207 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760f3b8-391b-4bf5-8c66-7d7fe3a58177 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:54.842734 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760f3b8-391b-4bf5-8c66-7d7fe3a58177 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:54.843215 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760f3b8-391b-4bf5-8c66-7d7fe3a58177 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:54.883893 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:54.919071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a6b3d-5b6d-41f5-a2d3-90f37849f3d0 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:54.935060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760f3b8-391b-4bf5-8c66-7d7fe3a58177 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:54.937237 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760f3b8-391b-4bf5-8c66-7d7fe3a58177 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:54.952891 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:55.023050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246a55b-2285-408e-95ba-10ccd0b473b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:55.024773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246a55b-2285-408e-95ba-10ccd0b473b7 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:55.024773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246a55b-2285-408e-95ba-10ccd0b473b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:55.025201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246a55b-2285-408e-95ba-10ccd0b473b7 None admin] PATH_INFO: `/v3/users/edcb474286f847c19da8fa8ad2250ee0` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:55.032987 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246a55b-2285-408e-95ba-10ccd0b473b7 None admin] RBAC: Authorizing `identity:get_user(user_id=edcb474286f847c19da8fa8ad2250ee0)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:55.035892 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246a55b-2285-408e-95ba-10ccd0b473b7 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:55.039743 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 192/371] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:57:54 2022] GET /identity/v3/users/edcb474286f847c19da8fa8ad2250ee0 => generated 293 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:55.048312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246a55b-2285-408e-95ba-10ccd0b473b7 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:55.066294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 180/372] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:57:55.067473 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760f3b8-391b-4bf5-8c66-7d7fe3a58177 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:55.179835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760f3b8-391b-4bf5-8c66-7d7fe3a58177 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:55.183465 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:55.220275 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6527b-a686-4176-a9c2-0bc4a7a6ecc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:55.221291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6527b-a686-4176-a9c2-0bc4a7a6ecc5 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:55.221778 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6527b-a686-4176-a9c2-0bc4a7a6ecc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:55.222254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6527b-a686-4176-a9c2-0bc4a7a6ecc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:55.223254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6527b-a686-4176-a9c2-0bc4a7a6ecc5 None admin] RBAC: Adding query filter params (role.id=46b7af1a00374c1989c960f2be43481e, user.id=edcb474286f847c19da8fa8ad2250ee0, scope.system=all) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:55.223802 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6527b-a686-4176-a9c2-0bc4a7a6ecc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:55.226212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6527b-a686-4176-a9c2-0bc4a7a6ecc5 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:55.240411 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 181/373] 10.222.0.93 () {56 vars in 1308 bytes} [Mon Aug 1 14:57:55 2022] GET /identity/v3/role_assignments?user.id=edcb474286f847c19da8fa8ad2250ee0&scope.system=all&role.id=46b7af1a00374c1989c960f2be43481e => generated 510 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:55.245953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6527b-a686-4176-a9c2-0bc4a7a6ecc5 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:55.280492 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246a55b-2285-408e-95ba-10ccd0b473b7 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:55.357443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca16e98-99d1-4909-9842-054562cf3c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:55.358439 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca16e98-99d1-4909-9842-054562cf3c1a None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:55.358439 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca16e98-99d1-4909-9842-054562cf3c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:55.358718 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca16e98-99d1-4909-9842-054562cf3c1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:55.398608 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6527b-a686-4176-a9c2-0bc4a7a6ecc5 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:55.409688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:55.457351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c67d4e-49d7-4fdf-850b-ab67a4c9a016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:55.458361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c67d4e-49d7-4fdf-850b-ab67a4c9a016 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:55.458850 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c67d4e-49d7-4fdf-850b-ab67a4c9a016 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:55.459375 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c67d4e-49d7-4fdf-850b-ab67a4c9a016 None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:55.460981 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c67d4e-49d7-4fdf-850b-ab67a4c9a016 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4838b7b9e4303a80d7610c6bef277)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:55.462726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c67d4e-49d7-4fdf-850b-ab67a4c9a016 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:55.464707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 182/374] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:55 2022] GET /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:55.473618 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c67d4e-49d7-4fdf-850b-ab67a4c9a016 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:55.488494 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca16e98-99d1-4909-9842-054562cf3c1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:55.488494 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca16e98-99d1-4909-9842-054562cf3c1a None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:55.534356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:55.610238 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 193/375] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:57:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:57:55.616966 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca16e98-99d1-4909-9842-054562cf3c1a None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:55.641832 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c67d4e-49d7-4fdf-850b-ab67a4c9a016 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:55.681040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:55.706828 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977f4d3-d4ef-489c-9d47-18d62ad56c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:55.707727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977f4d3-d4ef-489c-9d47-18d62ad56c56 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:55.708244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977f4d3-d4ef-489c-9d47-18d62ad56c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:55.708767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977f4d3-d4ef-489c-9d47-18d62ad56c56 None admin] PATH_INFO: `/v3/groups/56a3c992a6384e7f8444cc92b4180f64` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:55.713385 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b977f4d3-d4ef-489c-9d47-18d62ad56c56 None admin] RBAC: Authorizing `identity:get_group(group_id=56a3c992a6384e7f8444cc92b4180f64)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:55.715245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b977f4d3-d4ef-489c-9d47-18d62ad56c56 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:55.717228 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 183/376] 10.222.0.93 () {56 vars in 1180 bytes} [Mon Aug 1 14:57:55 2022] GET /identity/v3/groups/56a3c992a6384e7f8444cc92b4180f64 => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:55.727199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977f4d3-d4ef-489c-9d47-18d62ad56c56 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:55.735769 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca16e98-99d1-4909-9842-054562cf3c1a None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:55.810576 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a38bcf7-ee95-4c93-847e-436766ae40e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:55.811314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a38bcf7-ee95-4c93-847e-436766ae40e7 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:55.811594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a38bcf7-ee95-4c93-847e-436766ae40e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:55.811868 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a38bcf7-ee95-4c93-847e-436766ae40e7 None admin] PATH_INFO: `/v3/projects/28ee187de5744709bb65431943eb0200` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:55.815064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a38bcf7-ee95-4c93-847e-436766ae40e7 None admin] RBAC: Authorizing `identity:get_project(project_id=28ee187de5744709bb65431943eb0200)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:55.816587 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a38bcf7-ee95-4c93-847e-436766ae40e7 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:55.818566 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 194/377] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:55 2022] GET /identity/v3/projects/28ee187de5744709bb65431943eb0200 => generated 344 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:55.824631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a38bcf7-ee95-4c93-847e-436766ae40e7 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:55.963483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a38bcf7-ee95-4c93-847e-436766ae40e7 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:55.968861 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:55.977748 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977f4d3-d4ef-489c-9d47-18d62ad56c56 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:55.988700 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:56.010282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4a974-b640-4f22-a84a-593789afaac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:56.010943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4a974-b640-4f22-a84a-593789afaac3 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:56.011220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4a974-b640-4f22-a84a-593789afaac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:56.011496 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4a974-b640-4f22-a84a-593789afaac3 None admin] PATH_INFO: `/v3/groups/78b717d9e1c4435db7662d3cd8d14e9f` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:56.017771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff4a974-b640-4f22-a84a-593789afaac3 None admin] RBAC: Authorizing `identity:get_group(group_id=78b717d9e1c4435db7662d3cd8d14e9f)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:56.020167 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff4a974-b640-4f22-a84a-593789afaac3 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:56.022901 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 195/378] 10.222.0.93 () {56 vars in 1180 bytes} [Mon Aug 1 14:57:55 2022] GET /identity/v3/groups/78b717d9e1c4435db7662d3cd8d14e9f => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:56.024756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805b9b0-f50c-4a31-b713-8404219dbb04 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:56.025228 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805b9b0-f50c-4a31-b713-8404219dbb04 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:56.025660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805b9b0-f50c-4a31-b713-8404219dbb04 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:56.027570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 196/379] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:56.034346 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805b9b0-f50c-4a31-b713-8404219dbb04 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:56.079729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d75ceb-2bd9-49a0-868b-98423c629fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:56.080295 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d75ceb-2bd9-49a0-868b-98423c629fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:56.080676 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d75ceb-2bd9-49a0-868b-98423c629fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:56.080967 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d75ceb-2bd9-49a0-868b-98423c629fa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:56.081782 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d75ceb-2bd9-49a0-868b-98423c629fa9 None admin] RBAC: Adding query filter params (group.id=56a3c992a6384e7f8444cc92b4180f64, role.id=e880d7a2b8c74bb1beb5015723a4e3d7, scope.project.id=fcf4838b7b9e4303a80d7610c6bef277) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:56.082188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d75ceb-2bd9-49a0-868b-98423c629fa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:56.083911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d75ceb-2bd9-49a0-868b-98423c629fa9 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:56.097308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 184/380] 10.222.0.93 () {56 vars in 1376 bytes} [Mon Aug 1 14:57:55 2022] GET /identity/v3/role_assignments?group.id=56a3c992a6384e7f8444cc92b4180f64&scope.project.id=fcf4838b7b9e4303a80d7610c6bef277&role.id=e880d7a2b8c74bb1beb5015723a4e3d7 => generated 260 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:56.100098 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b7183-e1ed-4a8f-b212-bb52d8c38686 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:56.101420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b7183-e1ed-4a8f-b212-bb52d8c38686 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:56.102107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b7183-e1ed-4a8f-b212-bb52d8c38686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:56.144324 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-511b7183-e1ed-4a8f-b212-bb52d8c38686 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:56.147810 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-511b7183-e1ed-4a8f-b212-bb52d8c38686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:56.243192 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805b9b0-f50c-4a31-b713-8404219dbb04 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:56.298513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 185/381] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:56.315975 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6187e25-7bda-42c7-ab73-f4f28c8bdc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:56.316793 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6187e25-7bda-42c7-ab73-f4f28c8bdc66 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:56.317175 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6187e25-7bda-42c7-ab73-f4f28c8bdc66 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:56.317505 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6187e25-7bda-42c7-ab73-f4f28c8bdc66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:56.318457 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6187e25-7bda-42c7-ab73-f4f28c8bdc66 None admin] RBAC: Adding query filter params (group.id=78b717d9e1c4435db7662d3cd8d14e9f, role.id=5456887a306649bf80a9ea7bd4282e97, scope.project.id=28ee187de5744709bb65431943eb0200) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:56.318936 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6187e25-7bda-42c7-ab73-f4f28c8bdc66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:56.320901 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6187e25-7bda-42c7-ab73-f4f28c8bdc66 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:56.331200 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 197/382] 10.222.0.93 () {56 vars in 1376 bytes} [Mon Aug 1 14:57:56 2022] GET /identity/v3/role_assignments?group.id=78b717d9e1c4435db7662d3cd8d14e9f&scope.project.id=28ee187de5744709bb65431943eb0200&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 260 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:56.350524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b7183-e1ed-4a8f-b212-bb52d8c38686 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:56.510174 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:56.539250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b7183-e1ed-4a8f-b212-bb52d8c38686 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:56.597837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab2ce0-4bcf-4e8f-a013-a235305ef318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:56.598846 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab2ce0-4bcf-4e8f-a013-a235305ef318 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:56.599169 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab2ce0-4bcf-4e8f-a013-a235305ef318 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:56.599531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab2ce0-4bcf-4e8f-a013-a235305ef318 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:56.616366 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab2ce0-4bcf-4e8f-a013-a235305ef318 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:56.623127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-18ab2ce0-4bcf-4e8f-a013-a235305ef318 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:56.623127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:56.623127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:56.623127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:56.623127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:56.623127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:56.623127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:56.623127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:56.623127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:56.623127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:56.623127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:56.623127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:56.623127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:56.623127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:56.623127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:56.623127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:56.624088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:56.624088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:56.624088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:56.624088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:56.624088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:56.624088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:56.624088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:56.624088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:56.624088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:56.624088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:56.624088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:56.624088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:56.624088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:56.624088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:56.624088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:56.624088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:56.625006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:56.625006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:56.625006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:56.625006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:56.625006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:56.625006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:56.625006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:56.625006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:56.625006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:56.625006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:56.625006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:56.625006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:56.625878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 186/383] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:56.632925 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6187e25-7bda-42c7-ab73-f4f28c8bdc66 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:56.704746 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81d5ad-d21e-446c-af92-dded678e8d39 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:56.705088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81d5ad-d21e-446c-af92-dded678e8d39 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:56.705302 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81d5ad-d21e-446c-af92-dded678e8d39 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:56.706898 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 187/384] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:57:56.715170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499f0f3-e920-481e-84dc-ebfd46f841d8 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:56.715501 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499f0f3-e920-481e-84dc-ebfd46f841d8 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:56.715850 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499f0f3-e920-481e-84dc-ebfd46f841d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:56.728260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:56.744107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-c499f0f3-e920-481e-84dc-ebfd46f841d8 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:56.745481 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c499f0f3-e920-481e-84dc-ebfd46f841d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:56.831695 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6187e25-7bda-42c7-ab73-f4f28c8bdc66 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:56.878181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 188/385] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:56.917841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12572b1c-d0d3-4994-96ab-b13a9e359a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:56.918890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12572b1c-d0d3-4994-96ab-b13a9e359a15 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:56.919447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12572b1c-d0d3-4994-96ab-b13a9e359a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:56.919918 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12572b1c-d0d3-4994-96ab-b13a9e359a15 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:56.920903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12572b1c-d0d3-4994-96ab-b13a9e359a15 None admin] RBAC: Adding query filter params (name=member) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:56.922780 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12572b1c-d0d3-4994-96ab-b13a9e359a15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:56.923677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499f0f3-e920-481e-84dc-ebfd46f841d8 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:56.924741 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12572b1c-d0d3-4994-96ab-b13a9e359a15 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:56.947751 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 198/386] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:56.962547 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12572b1c-d0d3-4994-96ab-b13a9e359a15 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:57.098081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:57.135234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499f0f3-e920-481e-84dc-ebfd46f841d8 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:57.180502 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12572b1c-d0d3-4994-96ab-b13a9e359a15 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:57.184658 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdc225-8ece-4936-80ee-5006e25cf168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:57.185676 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdc225-8ece-4936-80ee-5006e25cf168 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:57.186218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdc225-8ece-4936-80ee-5006e25cf168 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:57.186695 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdc225-8ece-4936-80ee-5006e25cf168 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:57.188963 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:57.205524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdc225-8ece-4936-80ee-5006e25cf168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:57.216606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-18cdc225-8ece-4936-80ee-5006e25cf168 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:57.216606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:57.216606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:57.216606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:57.216606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:57.216606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:57.216606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:57.216606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:57.216606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:57.216606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:57.216606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:57.216606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:57.216606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:57.216606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:57.216606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:57.216606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:57.217584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:57.217584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:57.217584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:57.217584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:57.217584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:57.217584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:57.217584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:57.217584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:57.217584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:57.217584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:57.217584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:57.217584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:57.217584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:57.217584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:57.217584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:57.217584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:57.218395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:57.218395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:57.218395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:57.218395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:57.218395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:57.218395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:57.218395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:57.218395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:57.218395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:57.218395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:57.218395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:57.218395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:57.218395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 189/387] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:57:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:57.222249 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e4db8-3beb-465f-bd16-f1a9a694b102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:57.223041 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e4db8-3beb-465f-bd16-f1a9a694b102 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:57.223323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e4db8-3beb-465f-bd16-f1a9a694b102 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:57.223595 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e4db8-3beb-465f-bd16-f1a9a694b102 None admin] PATH_INFO: `/v3/groups/56a3c992a6384e7f8444cc92b4180f64` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:57.224996 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e4db8-3beb-465f-bd16-f1a9a694b102 None admin] RBAC: Authorizing `identity:get_group(group_id=56a3c992a6384e7f8444cc92b4180f64)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:57.227464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdc225-8ece-4936-80ee-5006e25cf168 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:57.232689 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e4db8-3beb-465f-bd16-f1a9a694b102 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:57.242630 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 199/388] 10.222.0.93 () {56 vars in 1180 bytes} [Mon Aug 1 14:57:56 2022] GET /identity/v3/groups/56a3c992a6384e7f8444cc92b4180f64 => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:57.247715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e4db8-3beb-465f-bd16-f1a9a694b102 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:57.376475 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e4db8-3beb-465f-bd16-f1a9a694b102 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:57.417555 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f58304-c381-4bce-818e-8436de8348c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:57.418671 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f58304-c381-4bce-818e-8436de8348c9 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:57.418954 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f58304-c381-4bce-818e-8436de8348c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:57.419225 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f58304-c381-4bce-818e-8436de8348c9 None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:57.420529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f58304-c381-4bce-818e-8436de8348c9 None admin] RBAC: Authorizing `identity:get_project(project_id=cc8867738cf24c5db418fa4bd4cd2c01)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:57.422266 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f58304-c381-4bce-818e-8436de8348c9 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:57.429182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 200/389] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:57 2022] GET /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:57.431613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f58304-c381-4bce-818e-8436de8348c9 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:57.508886 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:57.553686 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f58304-c381-4bce-818e-8436de8348c9 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:57.582526 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdc225-8ece-4936-80ee-5006e25cf168 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:57.598795 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9cfe3-e68d-4a8d-9d0c-20effcce10c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:57.600006 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9cfe3-e68d-4a8d-9d0c-20effcce10c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:57.600315 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9cfe3-e68d-4a8d-9d0c-20effcce10c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:57.600622 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9cfe3-e68d-4a8d-9d0c-20effcce10c6 None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01/groups/56a3c992a6384e7f8444cc92b4180f64/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:57.603377 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db9cfe3-e68d-4a8d-9d0c-20effcce10c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc8867738cf24c5db418fa4bd4cd2c01, group_id=56a3c992a6384e7f8444cc92b4180f64, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:57.604988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db9cfe3-e68d-4a8d-9d0c-20effcce10c6 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:57.629040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 201/390] 10.222.0.93 () {58 vars in 1442 bytes} [Mon Aug 1 14:57:57 2022] PUT /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01/groups/56a3c992a6384e7f8444cc92b4180f64/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:57:57.668044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fb218-1a78-4351-a9b7-935d06bf802e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:57.668828 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fb218-1a78-4351-a9b7-935d06bf802e None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:57.669327 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fb218-1a78-4351-a9b7-935d06bf802e None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:57.669854 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fb218-1a78-4351-a9b7-935d06bf802e None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:57.670627 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fb218-1a78-4351-a9b7-935d06bf802e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:57.670937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fb218-1a78-4351-a9b7-935d06bf802e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:57.672648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fb218-1a78-4351-a9b7-935d06bf802e None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:57.688544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 190/391] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:57:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:57:57.695124 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9cfe3-e68d-4a8d-9d0c-20effcce10c6 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:57.790849 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc6c36-1c1d-40b0-8b7b-556c59adc652 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:57.791196 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc6c36-1c1d-40b0-8b7b-556c59adc652 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:57.791421 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc6c36-1c1d-40b0-8b7b-556c59adc652 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:57.793183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 191/392] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:57.798907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c869a9-aaf6-4ac5-9825-60e084bc49a5 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:57.799259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c869a9-aaf6-4ac5-9825-60e084bc49a5 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:57.799536 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c869a9-aaf6-4ac5-9825-60e084bc49a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:57.807880 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:57.823906 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-b1c869a9-aaf6-4ac5-9825-60e084bc49a5 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:57.825799 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b1c869a9-aaf6-4ac5-9825-60e084bc49a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:57.887813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9cfe3-e68d-4a8d-9d0c-20effcce10c6 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:57.905120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:57.964285 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944196b9-4c36-49c0-b2fd-55e65fb53dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:57.964977 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944196b9-4c36-49c0-b2fd-55e65fb53dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:57.965270 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944196b9-4c36-49c0-b2fd-55e65fb53dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:57.965546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944196b9-4c36-49c0-b2fd-55e65fb53dd3 None admin] PATH_INFO: `/v3/groups/56a3c992a6384e7f8444cc92b4180f64` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:57.967410 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944196b9-4c36-49c0-b2fd-55e65fb53dd3 None admin] RBAC: Authorizing `identity:get_group(group_id=56a3c992a6384e7f8444cc92b4180f64)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:57.968901 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944196b9-4c36-49c0-b2fd-55e65fb53dd3 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:57.971266 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 202/393] 10.222.0.93 () {56 vars in 1180 bytes} [Mon Aug 1 14:57:57 2022] GET /identity/v3/groups/56a3c992a6384e7f8444cc92b4180f64 => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:57.973112 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 192/394] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:57:57.976859 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944196b9-4c36-49c0-b2fd-55e65fb53dd3 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:57.999499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c869a9-aaf6-4ac5-9825-60e084bc49a5 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:58.108759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944196b9-4c36-49c0-b2fd-55e65fb53dd3 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:58.149688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c869a9-aaf6-4ac5-9825-60e084bc49a5 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:58.178904 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dde0f8-1a09-4fbb-88b7-a61532de7c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:58.179553 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dde0f8-1a09-4fbb-88b7-a61532de7c41 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:58.179834 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dde0f8-1a09-4fbb-88b7-a61532de7c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:58.180104 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dde0f8-1a09-4fbb-88b7-a61532de7c41 None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:58.181630 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dde0f8-1a09-4fbb-88b7-a61532de7c41 None admin] RBAC: Authorizing `identity:get_project(project_id=cc8867738cf24c5db418fa4bd4cd2c01)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:58.184240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dde0f8-1a09-4fbb-88b7-a61532de7c41 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:58.188511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 203/395] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:57 2022] GET /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:58.195316 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dde0f8-1a09-4fbb-88b7-a61532de7c41 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:58.220596 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711cb93-7bbf-408e-97a7-ccd476f145b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:58.221367 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711cb93-7bbf-408e-97a7-ccd476f145b8 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:58.221650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711cb93-7bbf-408e-97a7-ccd476f145b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:58.221922 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711cb93-7bbf-408e-97a7-ccd476f145b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:58.225626 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:58.253653 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711cb93-7bbf-408e-97a7-ccd476f145b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:58.257821 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:58.260226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-d711cb93-7bbf-408e-97a7-ccd476f145b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:58.260226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:58.260226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:58.260226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:58.260226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:58.260226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:58.260226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:58.260226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:58.260226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:58.260226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:58.260226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:58.260226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:58.260226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:58.260226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:58.260226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:58.260226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:58.261272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:58.261272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:58.261272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:58.261272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:58.261272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:58.261272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:58.261272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:58.261272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:58.261272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:58.261272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:58.261272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:58.261272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:58.261272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:58.261272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:58.261272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:58.261272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:58.262104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:58.262104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:58.262104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:58.262104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:58.262104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:58.262104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:58.262104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:58.262104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:58.262104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:58.262104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:58.262104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:57:58.262104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:58.262104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 193/396] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:57:58.267879 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711cb93-7bbf-408e-97a7-ccd476f145b8 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:58.338358 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dde0f8-1a09-4fbb-88b7-a61532de7c41 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:58.385914 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775bf90-475f-4ea6-93ed-0c73814c862b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:58.386595 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775bf90-475f-4ea6-93ed-0c73814c862b None admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:58.386871 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775bf90-475f-4ea6-93ed-0c73814c862b None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:58.387144 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775bf90-475f-4ea6-93ed-0c73814c862b None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01/groups/56a3c992a6384e7f8444cc92b4180f64/roles/e880d7a2b8c74bb1beb5015723a4e3d7` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:58.389511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775bf90-475f-4ea6-93ed-0c73814c862b None admin] RBAC: Authorizing `identity:create_grant(project_id=cc8867738cf24c5db418fa4bd4cd2c01, group_id=56a3c992a6384e7f8444cc92b4180f64, role_id=e880d7a2b8c74bb1beb5015723a4e3d7)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:58.390966 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775bf90-475f-4ea6-93ed-0c73814c862b None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:58.407371 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 204/397] 10.222.0.93 () {58 vars in 1442 bytes} [Mon Aug 1 14:57:58 2022] PUT /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01/groups/56a3c992a6384e7f8444cc92b4180f64/roles/e880d7a2b8c74bb1beb5015723a4e3d7 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:57:58.477773 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711cb93-7bbf-408e-97a7-ccd476f145b8 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:58.522498 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894b764-5f42-4e4e-b5df-a08578bc946b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:58.524209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894b764-5f42-4e4e-b5df-a08578bc946b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:58.524209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894b764-5f42-4e4e-b5df-a08578bc946b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:58.524209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894b764-5f42-4e4e-b5df-a08578bc946b None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:58.524845 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894b764-5f42-4e4e-b5df-a08578bc946b None admin] RBAC: Adding query filter params (name=member) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:57:58.524845 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894b764-5f42-4e4e-b5df-a08578bc946b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:58.526224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894b764-5f42-4e4e-b5df-a08578bc946b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:58.545445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 194/398] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:57:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:58.550399 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775bf90-475f-4ea6-93ed-0c73814c862b None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:58.683870 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775bf90-475f-4ea6-93ed-0c73814c862b None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:58.685981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:58.737443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb43a3-df3c-4046-9f64-6922099d20a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:58.738717 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb43a3-df3c-4046-9f64-6922099d20a1 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:58.739235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb43a3-df3c-4046-9f64-6922099d20a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:58.739719 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb43a3-df3c-4046-9f64-6922099d20a1 None admin] PATH_INFO: `/v3/groups/56a3c992a6384e7f8444cc92b4180f64` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:58.741925 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb43a3-df3c-4046-9f64-6922099d20a1 None admin] RBAC: Authorizing `identity:get_group(group_id=56a3c992a6384e7f8444cc92b4180f64)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:58.743810 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb43a3-df3c-4046-9f64-6922099d20a1 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:58.745899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 205/399] 10.222.0.93 () {56 vars in 1180 bytes} [Mon Aug 1 14:57:58 2022] GET /identity/v3/groups/56a3c992a6384e7f8444cc92b4180f64 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:58.753550 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894b764-5f42-4e4e-b5df-a08578bc946b None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:58.872493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:58.935371 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894b764-5f42-4e4e-b5df-a08578bc946b None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:58.981531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac6c49-4b88-45ff-95a6-de5da149461a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:58.982185 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac6c49-4b88-45ff-95a6-de5da149461a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:58.982466 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac6c49-4b88-45ff-95a6-de5da149461a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:58.982742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac6c49-4b88-45ff-95a6-de5da149461a None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:58.983939 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac6c49-4b88-45ff-95a6-de5da149461a None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4838b7b9e4303a80d7610c6bef277)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:58.985422 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac6c49-4b88-45ff-95a6-de5da149461a None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:58.987407 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 195/400] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:57:58 2022] GET /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:57:58.993121 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb43a3-df3c-4046-9f64-6922099d20a1 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:59.144219 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb43a3-df3c-4046-9f64-6922099d20a1 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:59.191645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8fc02-1bf1-4209-ac59-87ae3d8cee03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:59.192248 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8fc02-1bf1-4209-ac59-87ae3d8cee03 None admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:59.192552 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8fc02-1bf1-4209-ac59-87ae3d8cee03 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:59.192829 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8fc02-1bf1-4209-ac59-87ae3d8cee03 None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277/groups/56a3c992a6384e7f8444cc92b4180f64/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:59.195342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a8fc02-1bf1-4209-ac59-87ae3d8cee03 None admin] RBAC: Authorizing `identity:create_grant(project_id=fcf4838b7b9e4303a80d7610c6bef277, group_id=56a3c992a6384e7f8444cc92b4180f64, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:57:59.196867 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a8fc02-1bf1-4209-ac59-87ae3d8cee03 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:57:59.215955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 206/401] 10.222.0.93 () {58 vars in 1442 bytes} [Mon Aug 1 14:57:58 2022] PUT /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277/groups/56a3c992a6384e7f8444cc92b4180f64/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:57:59.245999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe22f7-720e-427d-b727-6f5465f2ed07 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:59.246761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe22f7-720e-427d-b727-6f5465f2ed07 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:59.247001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe22f7-720e-427d-b727-6f5465f2ed07 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:59.248664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 196/402] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:59.253969 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a064d-610f-42be-a2ef-a54e48e928ee None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:59.254260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a064d-610f-42be-a2ef-a54e48e928ee None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:59.254540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a064d-610f-42be-a2ef-a54e48e928ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:59.290399 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-0b8a064d-610f-42be-a2ef-a54e48e928ee None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:59.291661 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0b8a064d-610f-42be-a2ef-a54e48e928ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:59.328623 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:59.402056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb6fb8-7518-4f8e-b620-041a666d21c6 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:59.402495 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb6fb8-7518-4f8e-b620-041a666d21c6 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:59.402915 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb6fb8-7518-4f8e-b620-041a666d21c6 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:59.407276 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 207/404] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:59.408157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 197/404] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:57:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:57:59.414242 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3f563-0b48-4f46-a6e8-b1e5b29d76eb None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:59.414922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3f563-0b48-4f46-a6e8-b1e5b29d76eb None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:59.415400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3f563-0b48-4f46-a6e8-b1e5b29d76eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:59.436304 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-e0c3f563-0b48-4f46-a6e8-b1e5b29d76eb None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:57:59.438093 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e0c3f563-0b48-4f46-a6e8-b1e5b29d76eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:57:59.447416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb6fb8-7518-4f8e-b620-041a666d21c6 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:59.528765 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 208/405] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:57:59.555365 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3f563-0b48-4f46-a6e8-b1e5b29d76eb None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:59.673281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb6fb8-7518-4f8e-b620-041a666d21c6 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:59.717219 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:59.725373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:57:59.735000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a412d-c20c-4d15-9d46-1261c4f341b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:59.736030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a412d-c20c-4d15-9d46-1261c4f341b0 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:59.736498 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a412d-c20c-4d15-9d46-1261c4f341b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:59.736968 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a412d-c20c-4d15-9d46-1261c4f341b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:59.746655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3f563-0b48-4f46-a6e8-b1e5b29d76eb None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:57:59.756819 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a412d-c20c-4d15-9d46-1261c4f341b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:59.766148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-a82a412d-c20c-4d15-9d46-1261c4f341b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:59.766148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:59.766148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:59.766148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:59.766148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:59.766148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:59.766148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:59.766148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:59.766148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:59.766148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:59.766148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:59.766148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:59.766148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:59.766148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:59.766148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:59.766148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:59.767126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:59.767126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:59.767126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:59.767126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:59.767126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:59.767126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:59.767126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:59.767126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:59.767126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:59.767126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:59.767126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:59.767126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:59.767126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:59.767126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:59.767126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:59.767126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:59.767936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:59.767936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:59.767936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:59.767936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:59.767936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:59.767936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:59.767936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:59.767936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:59.767936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:59.767936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:59.767936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:57:59.767936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:57:59.769135 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 198/406] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:57:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:57:59.782086 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a412d-c20c-4d15-9d46-1261c4f341b0 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:59.811152 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15413857-f2d9-4fe1-849a-c4436ebda022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:57:59.812240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15413857-f2d9-4fe1-849a-c4436ebda022 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:57:59.812825 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15413857-f2d9-4fe1-849a-c4436ebda022 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:57:59.813308 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15413857-f2d9-4fe1-849a-c4436ebda022 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:57:59.834164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15413857-f2d9-4fe1-849a-c4436ebda022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:57:59.844134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-15413857-f2d9-4fe1-849a-c4436ebda022 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:57:59.844134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:57:59.844134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:57:59.844134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:57:59.844134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:57:59.844134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:57:59.844134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:57:59.844134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:57:59.844134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:57:59.844134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:57:59.844134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:57:59.844134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:57:59.844134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:57:59.844134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:57:59.844134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:57:59.844134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:57:59.846495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:57:59.846495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:57:59.846495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:57:59.846495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:57:59.846495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:57:59.846495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:57:59.846495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:57:59.846495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:57:59.846495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:57:59.846495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:57:59.846495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:57:59.846495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:57:59.846495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:57:59.846495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:57:59.846495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:57:59.846495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:57:59.848577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:57:59.848577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:57:59.848577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:57:59.848577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:57:59.848577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:57:59.848577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:57:59.848577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:57:59.848577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:57:59.848577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:57:59.848577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:57:59.848577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:57:59.848577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:57:59.848577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 209/407] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:57:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:57:59.856203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15413857-f2d9-4fe1-849a-c4436ebda022 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:57:59.986131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a412d-c20c-4d15-9d46-1261c4f341b0 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:00.049009 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d8960-23ef-4358-92b0-a140f9e720fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:00.050006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d8960-23ef-4358-92b0-a140f9e720fb None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:00.050461 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d8960-23ef-4358-92b0-a140f9e720fb None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:00.050908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d8960-23ef-4358-92b0-a140f9e720fb None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:00.052143 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d8960-23ef-4358-92b0-a140f9e720fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:00.052677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d8960-23ef-4358-92b0-a140f9e720fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:00.055804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d8960-23ef-4358-92b0-a140f9e720fb None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:00.058317 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15413857-f2d9-4fe1-849a-c4436ebda022 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:00.071054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 199/408] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:57:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:00.076646 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d8960-23ef-4358-92b0-a140f9e720fb None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:00.104909 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900bc3f-ff68-4c5d-b107-0f72b8e1fcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:00.105597 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900bc3f-ff68-4c5d-b107-0f72b8e1fcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:00.105881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900bc3f-ff68-4c5d-b107-0f72b8e1fcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:00.106216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900bc3f-ff68-4c5d-b107-0f72b8e1fcd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:00.107040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900bc3f-ff68-4c5d-b107-0f72b8e1fcd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:00.107452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900bc3f-ff68-4c5d-b107-0f72b8e1fcd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:00.108981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900bc3f-ff68-4c5d-b107-0f72b8e1fcd4 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:00.120103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 210/409] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:57:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:58:00.126206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900bc3f-ff68-4c5d-b107-0f72b8e1fcd4 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:00.135181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:00.147476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:00.203615 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d8960-23ef-4358-92b0-a140f9e720fb None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:00.240480 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574ed52-9548-4347-a683-5f902974ec43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:00.241646 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574ed52-9548-4347-a683-5f902974ec43 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:00.241999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574ed52-9548-4347-a683-5f902974ec43 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:00.242201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574ed52-9548-4347-a683-5f902974ec43 None admin] PATH_INFO: `/v3/groups/56a3c992a6384e7f8444cc92b4180f64` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:00.243429 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1574ed52-9548-4347-a683-5f902974ec43 None admin] RBAC: Authorizing `identity:get_group(group_id=56a3c992a6384e7f8444cc92b4180f64)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:00.244920 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1574ed52-9548-4347-a683-5f902974ec43 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:00.247531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 200/410] 10.222.0.93 () {56 vars in 1180 bytes} [Mon Aug 1 14:58:00 2022] GET /identity/v3/groups/56a3c992a6384e7f8444cc92b4180f64 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:00.252144 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900bc3f-ff68-4c5d-b107-0f72b8e1fcd4 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:00.257173 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574ed52-9548-4347-a683-5f902974ec43 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:00.292945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09891c14-2673-43ca-b587-7990110b74ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:00.293906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09891c14-2673-43ca-b587-7990110b74ca None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:00.294343 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09891c14-2673-43ca-b587-7990110b74ca None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:00.294757 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09891c14-2673-43ca-b587-7990110b74ca None admin] PATH_INFO: `/v3/groups/78b717d9e1c4435db7662d3cd8d14e9f` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:00.296149 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09891c14-2673-43ca-b587-7990110b74ca None admin] RBAC: Authorizing `identity:get_group(group_id=78b717d9e1c4435db7662d3cd8d14e9f)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:00.297948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09891c14-2673-43ca-b587-7990110b74ca None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:00.300089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 211/411] 10.222.0.93 () {56 vars in 1180 bytes} [Mon Aug 1 14:58:00 2022] GET /identity/v3/groups/78b717d9e1c4435db7662d3cd8d14e9f => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:58:00.305706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09891c14-2673-43ca-b587-7990110b74ca None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:00.399272 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:00.422859 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09891c14-2673-43ca-b587-7990110b74ca None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:00.461498 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591b896-efae-45d5-8a5f-bab9cb3bdec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:00.462544 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591b896-efae-45d5-8a5f-bab9cb3bdec1 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:00.463025 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591b896-efae-45d5-8a5f-bab9cb3bdec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:00.463545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591b896-efae-45d5-8a5f-bab9cb3bdec1 None admin] PATH_INFO: `/v3/projects/28ee187de5744709bb65431943eb0200` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:00.465198 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591b896-efae-45d5-8a5f-bab9cb3bdec1 None admin] RBAC: Authorizing `identity:get_project(project_id=28ee187de5744709bb65431943eb0200)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:00.466958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591b896-efae-45d5-8a5f-bab9cb3bdec1 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:00.469280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 212/412] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:58:00 2022] GET /identity/v3/projects/28ee187de5744709bb65431943eb0200 => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:58:00.475377 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591b896-efae-45d5-8a5f-bab9cb3bdec1 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:00.504983 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574ed52-9548-4347-a683-5f902974ec43 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:00.543268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:00.595038 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c148509-0b87-4b3e-a2e7-931ac28a892f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:00.597134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c148509-0b87-4b3e-a2e7-931ac28a892f None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:00.601118 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c148509-0b87-4b3e-a2e7-931ac28a892f None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:00.601717 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c148509-0b87-4b3e-a2e7-931ac28a892f None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:00.603312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c148509-0b87-4b3e-a2e7-931ac28a892f None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4838b7b9e4303a80d7610c6bef277)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:00.607559 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c148509-0b87-4b3e-a2e7-931ac28a892f None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:00.609831 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 201/413] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:58:00 2022] GET /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277 => generated 302 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:00.616201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c148509-0b87-4b3e-a2e7-931ac28a892f None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:00.647601 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591b896-efae-45d5-8a5f-bab9cb3bdec1 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:00.723608 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bdd68-e55b-44fd-a057-a84c0a66c548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:00.724709 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bdd68-e55b-44fd-a057-a84c0a66c548 None admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:00.725765 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bdd68-e55b-44fd-a057-a84c0a66c548 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:00.726245 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bdd68-e55b-44fd-a057-a84c0a66c548 None admin] PATH_INFO: `/v3/projects/28ee187de5744709bb65431943eb0200/groups/78b717d9e1c4435db7662d3cd8d14e9f/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:00.728622 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614bdd68-e55b-44fd-a057-a84c0a66c548 None admin] RBAC: Authorizing `identity:create_grant(project_id=28ee187de5744709bb65431943eb0200, group_id=78b717d9e1c4435db7662d3cd8d14e9f, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:00.730318 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614bdd68-e55b-44fd-a057-a84c0a66c548 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:00.749831 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 213/414] 10.222.0.93 () {58 vars in 1442 bytes} [Mon Aug 1 14:58:00 2022] PUT /identity/v3/projects/28ee187de5744709bb65431943eb0200/groups/78b717d9e1c4435db7662d3cd8d14e9f/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:58:00.754693 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9d6b1-9fcc-4fa9-8845-228af61abd72 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:00.754693 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9d6b1-9fcc-4fa9-8845-228af61abd72 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:00.754693 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9d6b1-9fcc-4fa9-8845-228af61abd72 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:00.754693 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 214/415] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:00.761052 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b926f23-6263-408a-8d6e-54beaa3e60c2 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:00.761218 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b926f23-6263-408a-8d6e-54beaa3e60c2 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:00.761512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b926f23-6263-408a-8d6e-54beaa3e60c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:00.781973 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-7b926f23-6263-408a-8d6e-54beaa3e60c2 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:00.783214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-7b926f23-6263-408a-8d6e-54beaa3e60c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:00.800359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:00.819476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c148509-0b87-4b3e-a2e7-931ac28a892f None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:00.893075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 215/416] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:00.910227 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48020a1c-c217-4de1-a4c6-9fda15611f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:00.910969 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48020a1c-c217-4de1-a4c6-9fda15611f78 None admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:00.911255 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48020a1c-c217-4de1-a4c6-9fda15611f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:00.911531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48020a1c-c217-4de1-a4c6-9fda15611f78 None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277/groups/56a3c992a6384e7f8444cc92b4180f64/roles/e880d7a2b8c74bb1beb5015723a4e3d7` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:00.915190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48020a1c-c217-4de1-a4c6-9fda15611f78 None admin] RBAC: Authorizing `identity:create_grant(project_id=fcf4838b7b9e4303a80d7610c6bef277, group_id=56a3c992a6384e7f8444cc92b4180f64, role_id=e880d7a2b8c74bb1beb5015723a4e3d7)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:00.916754 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48020a1c-c217-4de1-a4c6-9fda15611f78 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:00.937731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 202/417] 10.222.0.93 () {58 vars in 1442 bytes} [Mon Aug 1 14:58:00 2022] PUT /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277/groups/56a3c992a6384e7f8444cc92b4180f64/roles/e880d7a2b8c74bb1beb5015723a4e3d7 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:58:00.949850 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b926f23-6263-408a-8d6e-54beaa3e60c2 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:01.092877 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b926f23-6263-408a-8d6e-54beaa3e60c2 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:01.156514 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79dd229-b425-4130-baa6-ec7b1bcee9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:01.158036 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79dd229-b425-4130-baa6-ec7b1bcee9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:01.158770 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79dd229-b425-4130-baa6-ec7b1bcee9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:01.159541 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79dd229-b425-4130-baa6-ec7b1bcee9c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:01.165131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cb998-5950-4f75-bff4-6a6c80c47fd2 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:01.165576 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cb998-5950-4f75-bff4-6a6c80c47fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:01.166002 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cb998-5950-4f75-bff4-6a6c80c47fd2 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:01.167756 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 203/418] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:01.173638 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae76457-4cf2-4d6c-b2c8-a44758b920a6 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:01.173881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae76457-4cf2-4d6c-b2c8-a44758b920a6 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:01.174101 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae76457-4cf2-4d6c-b2c8-a44758b920a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:01.175108 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:01.181228 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79dd229-b425-4130-baa6-ec7b1bcee9c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:01.186291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-b79dd229-b425-4130-baa6-ec7b1bcee9c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:58:01.186291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:01.186291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:01.186291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:01.186291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:01.186291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:01.186291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:01.186291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:01.186291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:01.186291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:01.186291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:01.186291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:01.186291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:01.186291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:01.186291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:01.186291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:01.187203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:01.187203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:01.187203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:01.187203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:01.187203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:01.187203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:01.187203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:01.187203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:01.187203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:01.187203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:01.187203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:01.187203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:01.187203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:01.187203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:01.187203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:01.187203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:01.188057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:01.188057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:01.188057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:01.188057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:01.188057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:01.188057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:01.188057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:01.188057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:01.188057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:01.188057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:01.188057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:58:01.188057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:01.188057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 216/419] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:58:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:01.193380 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79dd229-b425-4130-baa6-ec7b1bcee9c8 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:01.195910 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-7ae76457-4cf2-4d6c-b2c8-a44758b920a6 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:01.197159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-7ae76457-4cf2-4d6c-b2c8-a44758b920a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:01.243352 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:01.291190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 204/420] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:01.322203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79dd229-b425-4130-baa6-ec7b1bcee9c8 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:01.331603 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae76457-4cf2-4d6c-b2c8-a44758b920a6 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:01.361893 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abf4ba-65fc-4f1c-95fe-4de29fc70aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:01.362438 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abf4ba-65fc-4f1c-95fe-4de29fc70aac None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:01.362717 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abf4ba-65fc-4f1c-95fe-4de29fc70aac None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:01.362988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abf4ba-65fc-4f1c-95fe-4de29fc70aac None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:01.363974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38abf4ba-65fc-4f1c-95fe-4de29fc70aac None admin] RBAC: Adding query filter params (name=member) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:01.364278 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38abf4ba-65fc-4f1c-95fe-4de29fc70aac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:01.365747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38abf4ba-65fc-4f1c-95fe-4de29fc70aac None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:01.376543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 217/421] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:58:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:01.383662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abf4ba-65fc-4f1c-95fe-4de29fc70aac None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:01.465443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:01.493872 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae76457-4cf2-4d6c-b2c8-a44758b920a6 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:01.556964 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abf4ba-65fc-4f1c-95fe-4de29fc70aac None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:01.590098 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a72a1d-6aab-4b50-af2a-90fcc18a213a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:01.590834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a72a1d-6aab-4b50-af2a-90fcc18a213a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:01.591131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a72a1d-6aab-4b50-af2a-90fcc18a213a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:01.591451 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a72a1d-6aab-4b50-af2a-90fcc18a213a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:01.625996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a72a1d-6aab-4b50-af2a-90fcc18a213a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:01.641814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-f1a72a1d-6aab-4b50-af2a-90fcc18a213a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:58:01.641814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:01.641814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:01.641814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:01.641814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:01.641814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:01.641814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:01.641814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:01.641814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:01.641814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:01.641814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:01.641814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:01.641814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:01.641814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:01.641814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:01.641814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:01.642996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:01.642996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:01.642996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:01.642996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:01.642996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:01.642996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:01.642996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:01.642996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:01.642996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:01.642996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:01.642996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:01.642996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:01.642996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:01.642996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:01.642996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:01.642996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:01.643924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:01.643924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:01.643924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:01.643924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:01.643924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:01.643924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:01.643924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:01.643924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:01.643924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:01.643924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:01.643924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:58:01.643924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:01.645513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 205/422] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:58:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:58:01.655934 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda11e2-ed2c-4bca-98cd-5d237072b4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:01.656629 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda11e2-ed2c-4bca-98cd-5d237072b4bf None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:01.656911 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda11e2-ed2c-4bca-98cd-5d237072b4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:01.657182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda11e2-ed2c-4bca-98cd-5d237072b4bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:01.659577 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a72a1d-6aab-4b50-af2a-90fcc18a213a None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:01.725197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:01.820498 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda11e2-ed2c-4bca-98cd-5d237072b4bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:01.822000 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda11e2-ed2c-4bca-98cd-5d237072b4bf None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:01.886053 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a72a1d-6aab-4b50-af2a-90fcc18a213a None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:01.926932 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628c008-bbcb-4c16-b014-fdadab763330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:01.927562 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628c008-bbcb-4c16-b014-fdadab763330 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:01.927874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628c008-bbcb-4c16-b014-fdadab763330 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:01.928157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628c008-bbcb-4c16-b014-fdadab763330 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:01.928947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e628c008-bbcb-4c16-b014-fdadab763330 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:01.929254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e628c008-bbcb-4c16-b014-fdadab763330 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:01.930726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e628c008-bbcb-4c16-b014-fdadab763330 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:01.933402 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:01.941878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 206/423] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:01.958415 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628c008-bbcb-4c16-b014-fdadab763330 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:02.024797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 218/424] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:58:02.038226 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda11e2-ed2c-4bca-98cd-5d237072b4bf None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:02.091466 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:02.100905 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628c008-bbcb-4c16-b014-fdadab763330 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:02.139583 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f47c7c-21ae-41cc-96c0-c8b983cb863d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:02.140150 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f47c7c-21ae-41cc-96c0-c8b983cb863d None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:02.140428 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f47c7c-21ae-41cc-96c0-c8b983cb863d None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:02.140731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f47c7c-21ae-41cc-96c0-c8b983cb863d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:02.229768 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f47c7c-21ae-41cc-96c0-c8b983cb863d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:02.231683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f47c7c-21ae-41cc-96c0-c8b983cb863d None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:02.266373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda11e2-ed2c-4bca-98cd-5d237072b4bf None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:02.313707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349014f2-e84a-44fc-bdfc-55c2b0112fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:02.315798 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349014f2-e84a-44fc-bdfc-55c2b0112fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:02.316299 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349014f2-e84a-44fc-bdfc-55c2b0112fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:02.316775 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349014f2-e84a-44fc-bdfc-55c2b0112fa3 None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:02.318206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349014f2-e84a-44fc-bdfc-55c2b0112fa3 None admin] RBAC: Authorizing `identity:get_project(project_id=cc8867738cf24c5db418fa4bd4cd2c01)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:02.319915 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349014f2-e84a-44fc-bdfc-55c2b0112fa3 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:02.322774 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 219/425] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:58:02 2022] GET /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01 => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:02.328058 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81545539-2482-4aa0-9c5f-c66993ec8e10 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:02.328058 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81545539-2482-4aa0-9c5f-c66993ec8e10 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:02.328058 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81545539-2482-4aa0-9c5f-c66993ec8e10 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:02.329110 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 220/426] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:02.330213 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81545539-2482-4aa0-9c5f-c66993ec8e10 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:02.353962 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:02.368823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 207/427] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:58:02.369769 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:02.372026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b290fbc-fcf8-4665-bcc3-a40661095e53 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:02.372581 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b290fbc-fcf8-4665-bcc3-a40661095e53 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:02.373068 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b290fbc-fcf8-4665-bcc3-a40661095e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:02.394972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-3b290fbc-fcf8-4665-bcc3-a40661095e53 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:02.396013 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-3b290fbc-fcf8-4665-bcc3-a40661095e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:02.476926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81545539-2482-4aa0-9c5f-c66993ec8e10 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:02.484705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 208/428] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:02.485875 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b290fbc-fcf8-4665-bcc3-a40661095e53 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:02.528807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ef980-7466-446b-9df9-4217736eaa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:02.530110 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ef980-7466-446b-9df9-4217736eaa75 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:02.530616 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ef980-7466-446b-9df9-4217736eaa75 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:02.531566 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ef980-7466-446b-9df9-4217736eaa75 None admin] PATH_INFO: `/v3/groups/56a3c992a6384e7f8444cc92b4180f64` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:02.533378 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ef980-7466-446b-9df9-4217736eaa75 None admin] RBAC: Authorizing `identity:get_group(group_id=56a3c992a6384e7f8444cc92b4180f64)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:02.536214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ef980-7466-446b-9df9-4217736eaa75 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:02.544420 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 221/429] 10.222.0.93 () {56 vars in 1180 bytes} [Mon Aug 1 14:58:02 2022] GET /identity/v3/groups/56a3c992a6384e7f8444cc92b4180f64 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:02.544420 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ef980-7466-446b-9df9-4217736eaa75 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:02.642833 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b290fbc-fcf8-4665-bcc3-a40661095e53 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:02.671102 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:02.677309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:02.681682 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ef980-7466-446b-9df9-4217736eaa75 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:02.694888 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05378a66-69df-4f8d-890d-a660b5ced24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:02.695912 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05378a66-69df-4f8d-890d-a660b5ced24a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:02.697130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05378a66-69df-4f8d-890d-a660b5ced24a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:02.697626 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05378a66-69df-4f8d-890d-a660b5ced24a None admin] PATH_INFO: `/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:02.699495 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05378a66-69df-4f8d-890d-a660b5ced24a None admin] RBAC: Authorizing `identity:get_project(project_id=cc8867738cf24c5db418fa4bd4cd2c01)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:02.701649 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05378a66-69df-4f8d-890d-a660b5ced24a None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:02.703900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 209/430] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:58:02 2022] GET /identity/v3/projects/cc8867738cf24c5db418fa4bd4cd2c01 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:02.705063 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05378a66-69df-4f8d-890d-a660b5ced24a None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:02.737027 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74580c8e-25fe-46e9-9271-4020ed0a882a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:02.737784 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74580c8e-25fe-46e9-9271-4020ed0a882a None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:02.738069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74580c8e-25fe-46e9-9271-4020ed0a882a None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:02.738487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74580c8e-25fe-46e9-9271-4020ed0a882a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:02.752704 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74580c8e-25fe-46e9-9271-4020ed0a882a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:02.758010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-74580c8e-25fe-46e9-9271-4020ed0a882a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:58:02.758010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:02.758010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:02.758010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:02.758010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:02.758010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:02.758010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:02.758010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:02.758010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:02.758010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:02.758010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:02.758010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:02.758010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:02.758010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:02.758010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:02.758010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:02.758943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:02.758943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:02.758943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:02.758943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:02.758943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:02.758943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:02.758943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:02.758943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:02.758943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:02.758943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:02.758943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:02.758943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:02.758943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:02.758943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:02.758943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:02.758943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:02.759753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:02.759753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:02.759753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:02.759753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:02.759753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:02.759753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:02.759753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:02.759753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:02.759753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:02.759753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:02.759753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:58:02.759753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:02.759753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 222/431] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:58:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:02.760553 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74580c8e-25fe-46e9-9271-4020ed0a882a None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:02.849764 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05378a66-69df-4f8d-890d-a660b5ced24a None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:02.893177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f173ae0-655e-42b1-8030-cef9accba242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:02.893586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f173ae0-655e-42b1-8030-cef9accba242 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:02.893869 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f173ae0-655e-42b1-8030-cef9accba242 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:02.894136 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f173ae0-655e-42b1-8030-cef9accba242 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:02.894932 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f173ae0-655e-42b1-8030-cef9accba242 None admin] RBAC: Adding query filter params (group.id=56a3c992a6384e7f8444cc92b4180f64, role.id=c65b8ad9ade94760af715b3899bdd799, scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:02.895239 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f173ae0-655e-42b1-8030-cef9accba242 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:02.897011 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f173ae0-655e-42b1-8030-cef9accba242 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:02.911815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 210/432] 10.222.0.93 () {56 vars in 1376 bytes} [Mon Aug 1 14:58:02 2022] GET /identity/v3/role_assignments?group.id=56a3c992a6384e7f8444cc92b4180f64&scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01&role.id=c65b8ad9ade94760af715b3899bdd799 => generated 611 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:02.913618 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f173ae0-655e-42b1-8030-cef9accba242 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:02.915759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74580c8e-25fe-46e9-9271-4020ed0a882a None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:02.961080 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959b164-2391-4dbe-9a70-7de92950872f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:02.962315 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959b164-2391-4dbe-9a70-7de92950872f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:02.962890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959b164-2391-4dbe-9a70-7de92950872f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:02.963407 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959b164-2391-4dbe-9a70-7de92950872f None admin] PATH_INFO: `/v3/groups/56a3c992a6384e7f8444cc92b4180f64` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:02.965277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959b164-2391-4dbe-9a70-7de92950872f None admin] RBAC: Authorizing `identity:get_group(group_id=56a3c992a6384e7f8444cc92b4180f64)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:02.967260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959b164-2391-4dbe-9a70-7de92950872f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:02.970081 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 223/433] 10.222.0.93 () {56 vars in 1180 bytes} [Mon Aug 1 14:58:02 2022] GET /identity/v3/groups/56a3c992a6384e7f8444cc92b4180f64 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:02.985536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959b164-2391-4dbe-9a70-7de92950872f None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:03.019039 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:03.040494 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:03.080499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f173ae0-655e-42b1-8030-cef9accba242 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:03.133428 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd82e47-08d5-455c-9b10-e97e7c15a7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:03.134098 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd82e47-08d5-455c-9b10-e97e7c15a7a7 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:03.134565 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd82e47-08d5-455c-9b10-e97e7c15a7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:03.135037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd82e47-08d5-455c-9b10-e97e7c15a7a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:03.135811 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd82e47-08d5-455c-9b10-e97e7c15a7a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:03.136257 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd82e47-08d5-455c-9b10-e97e7c15a7a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:03.138165 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd82e47-08d5-455c-9b10-e97e7c15a7a7 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:03.157708 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 211/434] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:58:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:03.161064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd82e47-08d5-455c-9b10-e97e7c15a7a7 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:03.168656 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959b164-2391-4dbe-9a70-7de92950872f None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:03.229639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316e37e-47ec-4621-821e-d3b444683a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:03.230597 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316e37e-47ec-4621-821e-d3b444683a7e None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:03.230946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316e37e-47ec-4621-821e-d3b444683a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:03.231153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316e37e-47ec-4621-821e-d3b444683a7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:03.231936 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316e37e-47ec-4621-821e-d3b444683a7e None admin] RBAC: Adding query filter params (group.id=56a3c992a6384e7f8444cc92b4180f64, role.id=e880d7a2b8c74bb1beb5015723a4e3d7, scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:03.232247 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316e37e-47ec-4621-821e-d3b444683a7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:03.233808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316e37e-47ec-4621-821e-d3b444683a7e None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:03.242052 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 224/435] 10.222.0.93 () {56 vars in 1376 bytes} [Mon Aug 1 14:58:02 2022] GET /identity/v3/role_assignments?group.id=56a3c992a6384e7f8444cc92b4180f64&scope.project.id=cc8867738cf24c5db418fa4bd4cd2c01&role.id=e880d7a2b8c74bb1beb5015723a4e3d7 => generated 611 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:03.243314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bec544-c561-48c2-b476-408f1ef03cd7 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:03.243623 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bec544-c561-48c2-b476-408f1ef03cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:03.243879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bec544-c561-48c2-b476-408f1ef03cd7 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:03.248734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 225/436] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:03.258118 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2cdd8-aec4-4dbd-8677-e105711d5ebb None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:03.258731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2cdd8-aec4-4dbd-8677-e105711d5ebb None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:03.259007 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2cdd8-aec4-4dbd-8677-e105711d5ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:03.290886 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-36d2cdd8-aec4-4dbd-8677-e105711d5ebb None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:03.293915 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-36d2cdd8-aec4-4dbd-8677-e105711d5ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:03.314376 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd82e47-08d5-455c-9b10-e97e7c15a7a7 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:03.365759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:03.408798 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 226/437] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:03.410004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5ce45-9ffb-49ac-b7b2-75208f4ecf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:03.411142 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5ce45-9ffb-49ac-b7b2-75208f4ecf11 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:03.411655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5ce45-9ffb-49ac-b7b2-75208f4ecf11 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:03.412232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5ce45-9ffb-49ac-b7b2-75208f4ecf11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:03.433738 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2cdd8-aec4-4dbd-8677-e105711d5ebb None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:03.455736 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:03.545284 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5ce45-9ffb-49ac-b7b2-75208f4ecf11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:03.548124 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5ce45-9ffb-49ac-b7b2-75208f4ecf11 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:03.595292 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2cdd8-aec4-4dbd-8677-e105711d5ebb None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:03.648797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1372ee9-9200-4333-8d46-f552aead2254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:03.649660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1372ee9-9200-4333-8d46-f552aead2254 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:03.649938 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1372ee9-9200-4333-8d46-f552aead2254 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:03.650217 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1372ee9-9200-4333-8d46-f552aead2254 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:03.664542 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1372ee9-9200-4333-8d46-f552aead2254 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:03.670014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-f1372ee9-9200-4333-8d46-f552aead2254 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:58:03.670014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:03.670014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:03.670014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:03.670014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:03.670014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:03.670014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:03.670014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:03.670014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:03.670014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:03.670014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:03.670014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:03.670014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:03.670014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:03.670014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:03.670014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:03.671059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:03.671059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:03.671059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:03.671059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:03.671059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:03.671059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:03.671059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:03.671059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:03.671059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:03.671059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:03.671059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:03.671059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:03.671059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:03.671059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:03.671059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:03.671059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:03.671965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:03.671965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:03.671965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:03.671965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:03.671965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:03.671965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:03.671965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:03.671965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:03.671965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:03.671965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:03.671965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:58:03.671965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:03.673277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 227/438] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:58:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:58:03.678747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362e342-5395-45cf-99f0-88eb9dacf724 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:03.679279 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362e342-5395-45cf-99f0-88eb9dacf724 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:03.679450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362e342-5395-45cf-99f0-88eb9dacf724 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:03.681714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 228/439] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:03.682862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362e342-5395-45cf-99f0-88eb9dacf724 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:03.694902 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 212/440] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:58:03.696564 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8769af5-2323-4634-8926-4156b783f9a9 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:03.696859 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8769af5-2323-4634-8926-4156b783f9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:03.697138 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8769af5-2323-4634-8926-4156b783f9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:03.720132 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-d8769af5-2323-4634-8926-4156b783f9a9 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:03.721610 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d8769af5-2323-4634-8926-4156b783f9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:03.738783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:03.811280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:03.821997 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 213/441] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:03.822924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8769af5-2323-4634-8926-4156b783f9a9 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:03.846164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362e342-5395-45cf-99f0-88eb9dacf724 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:03.897502 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382efbe0-203e-4062-a2f6-b41bf7084609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:03.897502 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382efbe0-203e-4062-a2f6-b41bf7084609 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:03.897502 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382efbe0-203e-4062-a2f6-b41bf7084609 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:03.897919 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382efbe0-203e-4062-a2f6-b41bf7084609 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:03.898538 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382efbe0-203e-4062-a2f6-b41bf7084609 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:03.898737 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382efbe0-203e-4062-a2f6-b41bf7084609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:03.900151 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382efbe0-203e-4062-a2f6-b41bf7084609 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:03.909317 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 229/442] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:58:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:03.910166 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382efbe0-203e-4062-a2f6-b41bf7084609 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:03.958025 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8769af5-2323-4634-8926-4156b783f9a9 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:04.001164 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e623a-358f-460a-a918-3ac31810a137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:04.007090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e623a-358f-460a-a918-3ac31810a137 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:04.008220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e623a-358f-460a-a918-3ac31810a137 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:04.009612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e623a-358f-460a-a918-3ac31810a137 None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:04.011053 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e623a-358f-460a-a918-3ac31810a137 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4838b7b9e4303a80d7610c6bef277)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:04.012706 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e623a-358f-460a-a918-3ac31810a137 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:04.019876 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 214/443] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:58:03 2022] GET /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:04.019876 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e623a-358f-460a-a918-3ac31810a137 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:04.027581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382efbe0-203e-4062-a2f6-b41bf7084609 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:04.036820 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:04.066691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719d4fd-a030-4376-9629-1c7459f982cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:04.067226 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719d4fd-a030-4376-9629-1c7459f982cb None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:04.067504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719d4fd-a030-4376-9629-1c7459f982cb None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:04.067779 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719d4fd-a030-4376-9629-1c7459f982cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:04.076847 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:04.080900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719d4fd-a030-4376-9629-1c7459f982cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:04.086021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-b719d4fd-a030-4376-9629-1c7459f982cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:58:04.086021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:04.086021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:04.086021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:04.086021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:04.086021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:04.086021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:04.086021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:04.086021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:04.086021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:04.086021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:04.086021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:04.086021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:04.086021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:04.086021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:04.086021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:04.087089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:04.087089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:04.087089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:04.087089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:04.087089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:04.087089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:04.087089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:04.087089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:04.087089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:04.087089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:04.087089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:04.087089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:04.087089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:04.087089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:04.087089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:04.087089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:04.087990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:04.087990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:04.087990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:04.087990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:04.087990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:04.087990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:04.087990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:04.087990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:04.087990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:04.087990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:04.087990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:58:04.087990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:04.087990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 230/444] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:58:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:04.087990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719d4fd-a030-4376-9629-1c7459f982cb None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:04.145960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e623a-358f-460a-a918-3ac31810a137 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:04.189257 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0d6c4-e75d-4426-abe6-cea328bbfdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:04.189920 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0d6c4-e75d-4426-abe6-cea328bbfdaf None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:04.190203 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0d6c4-e75d-4426-abe6-cea328bbfdaf None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:04.190479 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0d6c4-e75d-4426-abe6-cea328bbfdaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:04.212966 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719d4fd-a030-4376-9629-1c7459f982cb None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:04.264999 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c8393-4def-422e-b2d3-b93aac5a70b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:04.265716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c8393-4def-422e-b2d3-b93aac5a70b5 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:04.266062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c8393-4def-422e-b2d3-b93aac5a70b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:04.266538 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c8393-4def-422e-b2d3-b93aac5a70b5 None admin] PATH_INFO: `/v3/groups/56a3c992a6384e7f8444cc92b4180f64` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:04.268841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c8393-4def-422e-b2d3-b93aac5a70b5 None admin] RBAC: Authorizing `identity:get_group(group_id=56a3c992a6384e7f8444cc92b4180f64)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:04.270589 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c8393-4def-422e-b2d3-b93aac5a70b5 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:04.272328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 231/445] 10.222.0.93 () {56 vars in 1180 bytes} [Mon Aug 1 14:58:04 2022] GET /identity/v3/groups/56a3c992a6384e7f8444cc92b4180f64 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:04.273143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c8393-4def-422e-b2d3-b93aac5a70b5 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:04.284700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0d6c4-e75d-4426-abe6-cea328bbfdaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:04.286348 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0d6c4-e75d-4426-abe6-cea328bbfdaf None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:04.306799 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:04.350481 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:04.399181 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c8393-4def-422e-b2d3-b93aac5a70b5 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:04.408526 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 215/446] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:58:04.409377 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0d6c4-e75d-4426-abe6-cea328bbfdaf None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:04.448744 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138b779-f3c0-473a-bf7d-d2b33bec903f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:04.449071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138b779-f3c0-473a-bf7d-d2b33bec903f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:04.449353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138b779-f3c0-473a-bf7d-d2b33bec903f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:04.449623 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138b779-f3c0-473a-bf7d-d2b33bec903f None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:04.450420 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138b779-f3c0-473a-bf7d-d2b33bec903f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:04.450692 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138b779-f3c0-473a-bf7d-d2b33bec903f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:04.452089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138b779-f3c0-473a-bf7d-d2b33bec903f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:04.466178 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 232/447] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:04.466178 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138b779-f3c0-473a-bf7d-d2b33bec903f None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:04.530983 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0d6c4-e75d-4426-abe6-cea328bbfdaf None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:04.550867 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:04.575924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26e1fc-5f5a-4fbd-8f3b-42dd33538eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:04.576640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26e1fc-5f5a-4fbd-8f3b-42dd33538eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:04.576926 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26e1fc-5f5a-4fbd-8f3b-42dd33538eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:04.577158 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26e1fc-5f5a-4fbd-8f3b-42dd33538eb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:04.577962 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26e1fc-5f5a-4fbd-8f3b-42dd33538eb2 None admin] RBAC: Adding query filter params (group.id=56a3c992a6384e7f8444cc92b4180f64, role.id=c65b8ad9ade94760af715b3899bdd799, scope.project.id=fcf4838b7b9e4303a80d7610c6bef277) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:04.578297 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26e1fc-5f5a-4fbd-8f3b-42dd33538eb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:04.579806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26e1fc-5f5a-4fbd-8f3b-42dd33538eb2 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:04.586209 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138b779-f3c0-473a-bf7d-d2b33bec903f None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:04.587323 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 216/448] 10.222.0.93 () {56 vars in 1376 bytes} [Mon Aug 1 14:58:04 2022] GET /identity/v3/role_assignments?group.id=56a3c992a6384e7f8444cc92b4180f64&scope.project.id=fcf4838b7b9e4303a80d7610c6bef277&role.id=c65b8ad9ade94760af715b3899bdd799 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:04.588890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26e1fc-5f5a-4fbd-8f3b-42dd33538eb2 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:04.627256 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:04.635502 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5814d7c2-e859-4303-ab8a-3ee2e95cb300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:04.636214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814d7c2-e859-4303-ab8a-3ee2e95cb300 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:04.636526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814d7c2-e859-4303-ab8a-3ee2e95cb300 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:04.636803 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814d7c2-e859-4303-ab8a-3ee2e95cb300 None admin] PATH_INFO: `/v3/projects/28ee187de5744709bb65431943eb0200` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:04.638136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5814d7c2-e859-4303-ab8a-3ee2e95cb300 None admin] RBAC: Authorizing `identity:get_project(project_id=28ee187de5744709bb65431943eb0200)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:04.639732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5814d7c2-e859-4303-ab8a-3ee2e95cb300 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:04.641624 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 233/449] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:58:04 2022] GET /identity/v3/projects/28ee187de5744709bb65431943eb0200 => generated 344 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:04.647765 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5814d7c2-e859-4303-ab8a-3ee2e95cb300 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:04.737610 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26e1fc-5f5a-4fbd-8f3b-42dd33538eb2 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:04.790867 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1fd57-1a3c-40b7-9011-c526bc7a2b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:04.791531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1fd57-1a3c-40b7-9011-c526bc7a2b3d None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:04.791811 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1fd57-1a3c-40b7-9011-c526bc7a2b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:04.792084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1fd57-1a3c-40b7-9011-c526bc7a2b3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:04.795682 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5814d7c2-e859-4303-ab8a-3ee2e95cb300 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:04.840334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d56b2-08ab-4fc6-b605-a26a81859ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:04.841187 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d56b2-08ab-4fc6-b605-a26a81859ddc None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:04.842360 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d56b2-08ab-4fc6-b605-a26a81859ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:04.842643 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d56b2-08ab-4fc6-b605-a26a81859ddc None admin] PATH_INFO: `/v3/groups/78b717d9e1c4435db7662d3cd8d14e9f` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:04.843886 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d56b2-08ab-4fc6-b605-a26a81859ddc None admin] RBAC: Authorizing `identity:get_group(group_id=78b717d9e1c4435db7662d3cd8d14e9f)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:04.845599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d56b2-08ab-4fc6-b605-a26a81859ddc None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:04.848581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 234/450] 10.222.0.93 () {56 vars in 1180 bytes} [Mon Aug 1 14:58:04 2022] GET /identity/v3/groups/78b717d9e1c4435db7662d3cd8d14e9f => generated 229 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:58:04.854691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d56b2-08ab-4fc6-b605-a26a81859ddc None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:04.867927 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:04.889379 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1fd57-1a3c-40b7-9011-c526bc7a2b3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:04.890751 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1fd57-1a3c-40b7-9011-c526bc7a2b3d None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:04.941144 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:04.997370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d56b2-08ab-4fc6-b605-a26a81859ddc None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:05.018089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 217/451] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:58:05.024297 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1fd57-1a3c-40b7-9011-c526bc7a2b3d None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:05.034915 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff8cb2-db74-485c-84fe-fa29c080d1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:05.035538 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff8cb2-db74-485c-84fe-fa29c080d1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:05.035815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff8cb2-db74-485c-84fe-fa29c080d1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:05.036090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff8cb2-db74-485c-84fe-fa29c080d1d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:05.036910 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff8cb2-db74-485c-84fe-fa29c080d1d2 None admin] RBAC: Adding query filter params (group.id=78b717d9e1c4435db7662d3cd8d14e9f, role.id=5456887a306649bf80a9ea7bd4282e97, scope.project.id=28ee187de5744709bb65431943eb0200) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:05.037792 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff8cb2-db74-485c-84fe-fa29c080d1d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:05.039259 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff8cb2-db74-485c-84fe-fa29c080d1d2 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:05.046235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 235/452] 10.222.0.93 () {56 vars in 1376 bytes} [Mon Aug 1 14:58:04 2022] GET /identity/v3/role_assignments?group.id=78b717d9e1c4435db7662d3cd8d14e9f&scope.project.id=28ee187de5744709bb65431943eb0200&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:58:05.115340 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:05.142805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1fd57-1a3c-40b7-9011-c526bc7a2b3d None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:05.192853 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5d749-1d4f-48ba-9201-57e5cef4f33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:05.194060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5d749-1d4f-48ba-9201-57e5cef4f33d None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:05.194307 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5d749-1d4f-48ba-9201-57e5cef4f33d None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:05.194582 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5d749-1d4f-48ba-9201-57e5cef4f33d None admin] PATH_INFO: `/v3/projects/fcf4838b7b9e4303a80d7610c6bef277` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:05.195852 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5d749-1d4f-48ba-9201-57e5cef4f33d None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4838b7b9e4303a80d7610c6bef277)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:05.197856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5d749-1d4f-48ba-9201-57e5cef4f33d None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:05.199574 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 218/453] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:58:05 2022] GET /identity/v3/projects/fcf4838b7b9e4303a80d7610c6bef277 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:58:05.205333 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff8cb2-db74-485c-84fe-fa29c080d1d2 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:05.309647 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff8cb2-db74-485c-84fe-fa29c080d1d2 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:05.337549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:05.343591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caae700-0faf-439e-87a9-a35ef2cda144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:05.344227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caae700-0faf-439e-87a9-a35ef2cda144 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:05.344534 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caae700-0faf-439e-87a9-a35ef2cda144 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:05.344813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caae700-0faf-439e-87a9-a35ef2cda144 None admin] PATH_INFO: `/v3/groups/56a3c992a6384e7f8444cc92b4180f64` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:05.345945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caae700-0faf-439e-87a9-a35ef2cda144 None admin] RBAC: Authorizing `identity:get_group(group_id=56a3c992a6384e7f8444cc92b4180f64)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:05.347385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caae700-0faf-439e-87a9-a35ef2cda144 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:05.349009 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 236/454] 10.222.0.93 () {56 vars in 1180 bytes} [Mon Aug 1 14:58:05 2022] GET /identity/v3/groups/56a3c992a6384e7f8444cc92b4180f64 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:05.354932 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5d749-1d4f-48ba-9201-57e5cef4f33d None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:05.453329 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5d749-1d4f-48ba-9201-57e5cef4f33d None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:05.486689 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56165c3-aa98-45e7-9b9b-cfa53095d064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:05.487289 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56165c3-aa98-45e7-9b9b-cfa53095d064 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:05.487567 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56165c3-aa98-45e7-9b9b-cfa53095d064 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:05.487841 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56165c3-aa98-45e7-9b9b-cfa53095d064 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:05.488644 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56165c3-aa98-45e7-9b9b-cfa53095d064 None admin] RBAC: Adding query filter params (group.id=56a3c992a6384e7f8444cc92b4180f64, role.id=e880d7a2b8c74bb1beb5015723a4e3d7, scope.project.id=fcf4838b7b9e4303a80d7610c6bef277) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:05.488948 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56165c3-aa98-45e7-9b9b-cfa53095d064 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:05.490923 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56165c3-aa98-45e7-9b9b-cfa53095d064 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:05.498264 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 219/455] 10.222.0.93 () {56 vars in 1376 bytes} [Mon Aug 1 14:58:05 2022] GET /identity/v3/role_assignments?group.id=56a3c992a6384e7f8444cc92b4180f64&scope.project.id=fcf4838b7b9e4303a80d7610c6bef277&role.id=e880d7a2b8c74bb1beb5015723a4e3d7 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:08.584427 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af4f4f-22fb-4e03-bc8f-9ac08d133302 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:08.585662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af4f4f-22fb-4e03-bc8f-9ac08d133302 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:08.586173 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af4f4f-22fb-4e03-bc8f-9ac08d133302 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:08.589094 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 237/456] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:08.595570 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f758b3-c3f3-4776-8115-dc893c9af484 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:08.596124 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f758b3-c3f3-4776-8115-dc893c9af484 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:08.596230 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f758b3-c3f3-4776-8115-dc893c9af484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:08.625246 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-d0f758b3-c3f3-4776-8115-dc893c9af484 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:08.626474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d0f758b3-c3f3-4776-8115-dc893c9af484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:08.639088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:08.731967 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 220/457] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:08.773263 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af4f4f-22fb-4e03-bc8f-9ac08d133302 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:08.785508 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca1aa1-8915-4496-8f36-04475c88f61e None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:08.788190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca1aa1-8915-4496-8f36-04475c88f61e None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:08.788785 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca1aa1-8915-4496-8f36-04475c88f61e None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:08.792356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 221/458] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:08.795914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afc245-e4e4-4b59-9321-ace31b39a698 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:08.796487 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afc245-e4e4-4b59-9321-ace31b39a698 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:08.796991 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afc245-e4e4-4b59-9321-ace31b39a698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:08.828279 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-21afc245-e4e4-4b59-9321-ace31b39a698 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:08.829868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-21afc245-e4e4-4b59-9321-ace31b39a698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:08.924497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 222/459] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:58:08.926338 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6be9b-bab2-4d72-b28a-f44225fbbf8c None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:08.926923 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6be9b-bab2-4d72-b28a-f44225fbbf8c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:08.927424 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6be9b-bab2-4d72-b28a-f44225fbbf8c None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:08.929839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 223/460] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:08.935989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea201c-46a1-42d5-abcd-6a655e140985 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:08.936725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea201c-46a1-42d5-abcd-6a655e140985 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:08.937288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea201c-46a1-42d5-abcd-6a655e140985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:08.943639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af4f4f-22fb-4e03-bc8f-9ac08d133302 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:08.972591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-a9ea201c-46a1-42d5-abcd-6a655e140985 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:08.974245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a9ea201c-46a1-42d5-abcd-6a655e140985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:08.992402 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770f56c-ad7d-4979-9692-cb1241ecd813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:08.994450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770f56c-ad7d-4979-9692-cb1241ecd813 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:08.994958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770f56c-ad7d-4979-9692-cb1241ecd813 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:08.995395 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770f56c-ad7d-4979-9692-cb1241ecd813 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:09.026969 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770f56c-ad7d-4979-9692-cb1241ecd813 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:09.029393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770f56c-ad7d-4979-9692-cb1241ecd813 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:09.068957 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 224/461] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:58:09.070327 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea201c-46a1-42d5-abcd-6a655e140985 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:09.070574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-d770f56c-ad7d-4979-9692-cb1241ecd813 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:09.070574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:09.070574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:09.070574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:09.070574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:09.070574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:09.070574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:09.070574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:09.070574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:09.070574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:09.070574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:09.070574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:09.070574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:09.070574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:09.070574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:09.071437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:09.071437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:09.071437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:09.071437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:09.071437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:09.071437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:09.071437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:09.071437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:09.071437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:09.071437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:09.071437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:09.071437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:09.071437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:09.071437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:09.071437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:09.071437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:09.071437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:09.072322 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:09.072322 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:09.072322 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:09.072322 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:09.073143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 238/462] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:09.084490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:09.084685 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770f56c-ad7d-4979-9692-cb1241ecd813 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:09.184302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:09.233593 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea201c-46a1-42d5-abcd-6a655e140985 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:09.253278 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770f56c-ad7d-4979-9692-cb1241ecd813 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:09.293303 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988527a-2c83-43b3-9350-9c1f0dde8870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:09.294266 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988527a-2c83-43b3-9350-9c1f0dde8870 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:09.294741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988527a-2c83-43b3-9350-9c1f0dde8870 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:09.294907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff1213-4639-49dc-b416-e9762b5ef65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:09.295220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988527a-2c83-43b3-9350-9c1f0dde8870 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:09.295481 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff1213-4639-49dc-b416-e9762b5ef65c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:09.295810 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff1213-4639-49dc-b416-e9762b5ef65c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:09.296134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff1213-4639-49dc-b416-e9762b5ef65c None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:09.296859 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff1213-4639-49dc-b416-e9762b5ef65c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:09.297167 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff1213-4639-49dc-b416-e9762b5ef65c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:09.298580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff1213-4639-49dc-b416-e9762b5ef65c None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:09.317403 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 239/463] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:09.317603 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988527a-2c83-43b3-9350-9c1f0dde8870 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:09.317699 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22657e9e-b329-4033-975d-33e2bde0720e None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:09.317699 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22657e9e-b329-4033-975d-33e2bde0720e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:09.317699 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22657e9e-b329-4033-975d-33e2bde0720e None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:09.317947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988527a-2c83-43b3-9350-9c1f0dde8870 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:09.318028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 240/464] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:09.318028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22657e9e-b329-4033-975d-33e2bde0720e None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:09.344256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-7988527a-2c83-43b3-9350-9c1f0dde8870 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:09.344256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:09.344256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:09.344256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:09.344256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:09.344256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:09.344256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:09.344256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:09.344256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:09.344256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:09.344256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:09.344256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:09.344256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:09.344256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:09.344256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:09.345900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:09.345900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:09.345900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:09.345900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:09.345900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:09.345900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:09.345900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:09.345900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:09.345900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:09.345900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:09.345900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:09.345900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:09.345900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:09.345900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:09.345900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:09.345900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:09.345900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:09.347605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:09.347605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:09.347605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:09.347605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:09.347605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 225/465] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:09.347605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988527a-2c83-43b3-9350-9c1f0dde8870 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:09.413146 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:09.448240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22657e9e-b329-4033-975d-33e2bde0720e None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:09.460142 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:09.463157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988527a-2c83-43b3-9350-9c1f0dde8870 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:09.489789 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00a455-8c7d-42fc-8983-4b828e745b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:09.490947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00a455-8c7d-42fc-8983-4b828e745b5f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:09.491499 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00a455-8c7d-42fc-8983-4b828e745b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:09.492074 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00a455-8c7d-42fc-8983-4b828e745b5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:09.507060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00a455-8c7d-42fc-8983-4b828e745b5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:09.508778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00a455-8c7d-42fc-8983-4b828e745b5f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:09.513971 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf439f2-2e77-44c1-8b71-7ed852347dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:09.514412 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf439f2-2e77-44c1-8b71-7ed852347dd7 None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:09.514697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf439f2-2e77-44c1-8b71-7ed852347dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:09.514975 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf439f2-2e77-44c1-8b71-7ed852347dd7 None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:09.515658 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf439f2-2e77-44c1-8b71-7ed852347dd7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:09.518272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf439f2-2e77-44c1-8b71-7ed852347dd7 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:09.525235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-4b00a455-8c7d-42fc-8983-4b828e745b5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:09.525235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:09.525235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:09.525235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:09.525235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:09.525235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:09.525235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:09.525235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:09.525235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:09.525235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:09.525235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:09.525235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:09.525235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:09.525235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:09.525235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:09.526902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:09.526902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:09.526902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:09.526902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:09.526902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:09.526902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:09.526902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:09.526902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:09.526902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:09.526902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:09.526902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:09.526902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:09.526902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:09.526902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:09.526902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:09.526902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:09.526902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:09.529358 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:09.529358 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:09.529358 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:09.529358 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:09.531451 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 241/466] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:09.533651 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0293c7-5dc0-4ebf-8035-aed760ba4659 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:09.534096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0293c7-5dc0-4ebf-8035-aed760ba4659 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:09.534481 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0293c7-5dc0-4ebf-8035-aed760ba4659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:09.556878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 226/467] 10.222.0.93 () {60 vars in 1132 bytes} [Mon Aug 1 14:58:09 2022] POST /identity/v3/users => generated 248 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:58:09.556878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf439f2-2e77-44c1-8b71-7ed852347dd7 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:09.557244 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-4a0293c7-5dc0-4ebf-8035-aed760ba4659 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:09.557396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4a0293c7-5dc0-4ebf-8035-aed760ba4659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:09.697648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf439f2-2e77-44c1-8b71-7ed852347dd7 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:09.704230 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 242/468] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:09.705767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0293c7-5dc0-4ebf-8035-aed760ba4659 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:09.711150 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:09.747686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09739f51-e392-494e-8d6c-376aa36eb260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:09.748677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09739f51-e392-494e-8d6c-376aa36eb260 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:09.749110 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09739f51-e392-494e-8d6c-376aa36eb260 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:09.749534 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09739f51-e392-494e-8d6c-376aa36eb260 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:09.750394 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09739f51-e392-494e-8d6c-376aa36eb260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:09.750880 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09739f51-e392-494e-8d6c-376aa36eb260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:09.752610 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09739f51-e392-494e-8d6c-376aa36eb260 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:09.778069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 227/469] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:09.779242 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09739f51-e392-494e-8d6c-376aa36eb260 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:09.872524 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:09.893887 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0293c7-5dc0-4ebf-8035-aed760ba4659 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:09.952744 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a850b7-44bb-4700-a7fd-f4b750c7d42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:09.954190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a850b7-44bb-4700-a7fd-f4b750c7d42e None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:09.954705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a850b7-44bb-4700-a7fd-f4b750c7d42e None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:09.955920 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a850b7-44bb-4700-a7fd-f4b750c7d42e None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:09.957225 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a850b7-44bb-4700-a7fd-f4b750c7d42e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:09.957842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a850b7-44bb-4700-a7fd-f4b750c7d42e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:09.959554 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a850b7-44bb-4700-a7fd-f4b750c7d42e None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:09.972753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09739f51-e392-494e-8d6c-376aa36eb260 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:09.980014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 243/470] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:09.981192 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a850b7-44bb-4700-a7fd-f4b750c7d42e None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:10.035868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5d6ca-dd3e-4100-9642-cb241a022393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:10.037750 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5d6ca-dd3e-4100-9642-cb241a022393 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:10.038232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5d6ca-dd3e-4100-9642-cb241a022393 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:10.038689 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5d6ca-dd3e-4100-9642-cb241a022393 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:10.054695 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5d6ca-dd3e-4100-9642-cb241a022393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:10.056651 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5d6ca-dd3e-4100-9642-cb241a022393 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:10.081890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-b7c5d6ca-dd3e-4100-9642-cb241a022393 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:10.081890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:10.081890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:10.081890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:10.081890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:10.081890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:10.081890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:10.081890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:10.081890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:10.081890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:10.081890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:10.081890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:10.081890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:10.081890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:10.081890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:10.082963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:10.082963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:10.082963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:10.082963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:10.082963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:10.082963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:10.082963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:10.082963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:10.082963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:10.082963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:10.082963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:10.082963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:10.082963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:10.082963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:10.082963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:10.082963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:10.082963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:10.084399 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:10.084399 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:10.084399 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:10.084399 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:10.087896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 228/471] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:10.089337 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5d6ca-dd3e-4100-9642-cb241a022393 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:10.142850 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:10.201777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a850b7-44bb-4700-a7fd-f4b750c7d42e None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:10.259824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c412b2-35fa-415a-a654-a36851f2b5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:10.261951 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c412b2-35fa-415a-a654-a36851f2b5a7 None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:10.262155 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c412b2-35fa-415a-a654-a36851f2b5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:10.262708 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c412b2-35fa-415a-a654-a36851f2b5a7 None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:10.264064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c412b2-35fa-415a-a654-a36851f2b5a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:10.267340 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c412b2-35fa-415a-a654-a36851f2b5a7 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:10.271899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5d6ca-dd3e-4100-9642-cb241a022393 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:10.309145 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 244/472] 10.222.0.93 () {60 vars in 1132 bytes} [Mon Aug 1 14:58:09 2022] POST /identity/v3/users => generated 248 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:58:10.316073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c412b2-35fa-415a-a654-a36851f2b5a7 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:10.339128 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:10.345273 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1d138-5e22-451e-a4ca-995417ea2eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:10.346989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1d138-5e22-451e-a4ca-995417ea2eea None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:10.348043 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1d138-5e22-451e-a4ca-995417ea2eea None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:10.352763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1d138-5e22-451e-a4ca-995417ea2eea None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:10.353963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1d138-5e22-451e-a4ca-995417ea2eea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:10.355618 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1d138-5e22-451e-a4ca-995417ea2eea None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:10.408270 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 229/473] 10.222.0.93 () {60 vars in 1132 bytes} [Mon Aug 1 14:58:10 2022] POST /identity/v3/users => generated 246 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:58:10.560958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c412b2-35fa-415a-a654-a36851f2b5a7 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:10.599656 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d191a5a-accc-47e5-bec7-e2b49781d3f9 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:10.604899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d191a5a-accc-47e5-bec7-e2b49781d3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:10.606119 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d191a5a-accc-47e5-bec7-e2b49781d3f9 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:10.609296 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 230/474] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:10 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:58:10.620328 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e11069-9452-4dd6-b26a-5a596b8af38e None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:10.621640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e11069-9452-4dd6-b26a-5a596b8af38e None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:10.622477 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e11069-9452-4dd6-b26a-5a596b8af38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:10.633184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4618cf9-185f-4dec-86cc-51cb5324c51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:10.633184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4618cf9-185f-4dec-86cc-51cb5324c51e None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:10.633184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4618cf9-185f-4dec-86cc-51cb5324c51e None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:10.633184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4618cf9-185f-4dec-86cc-51cb5324c51e None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:10.633842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4618cf9-185f-4dec-86cc-51cb5324c51e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:10.633842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4618cf9-185f-4dec-86cc-51cb5324c51e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:10.638673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4618cf9-185f-4dec-86cc-51cb5324c51e None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:10.663167 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 245/475] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:10.669907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4618cf9-185f-4dec-86cc-51cb5324c51e None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:10.675452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-82e11069-9452-4dd6-b26a-5a596b8af38e None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:10.678826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-82e11069-9452-4dd6-b26a-5a596b8af38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:10.807739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:10.824590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 231/476] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:10.853021 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e11069-9452-4dd6-b26a-5a596b8af38e None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:10.872532 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:10.880500 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4618cf9-185f-4dec-86cc-51cb5324c51e None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:10.943797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88c33a-7aa4-43c2-b0eb-bebede930c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:10.944910 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88c33a-7aa4-43c2-b0eb-bebede930c63 None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:10.945408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88c33a-7aa4-43c2-b0eb-bebede930c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:10.945895 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88c33a-7aa4-43c2-b0eb-bebede930c63 None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:10.949454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88c33a-7aa4-43c2-b0eb-bebede930c63 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:10.949903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88c33a-7aa4-43c2-b0eb-bebede930c63 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:10.991041 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 246/477] 10.222.0.93 () {60 vars in 1132 bytes} [Mon Aug 1 14:58:10 2022] POST /identity/v3/users => generated 249 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:58:11.085893 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e11069-9452-4dd6-b26a-5a596b8af38e None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:11.161028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcc199-d332-4cfc-9ac7-b33583792280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:11.162161 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcc199-d332-4cfc-9ac7-b33583792280 None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:11.162676 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcc199-d332-4cfc-9ac7-b33583792280 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:11.163182 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcc199-d332-4cfc-9ac7-b33583792280 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:11.164737 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcc199-d332-4cfc-9ac7-b33583792280 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:11.167538 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcc199-d332-4cfc-9ac7-b33583792280 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:11.181380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.sql.core [None req-62bcc199-d332-4cfc-9ac7-b33583792280 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 14:58:11.181380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 14:58:11.181380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [parameters: {'id': '6f3ad88546df4564b3f2a60e3af41d4d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 14:58:11.181380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80411) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 14:58:11.218657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-62bcc199-d332-4cfc-9ac7-b33583792280 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 14:58:11.218657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:11.218657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 14:58:11.218657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 14:58:11.218657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 14:58:11.218657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 14:58:11.218657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 14:58:11.218657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 14:58:11.218657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 14:58:11.218657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application conn.query(q) Aug 01 14:58:11.218657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 14:58:11.218657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 14:58:11.218657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 14:58:11.218657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application result.read() Aug 01 14:58:11.218657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 14:58:11.219698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 14:58:11.219698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 14:58:11.219698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 14:58:11.219698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 14:58:11.219698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 14:58:11.219698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 14:58:11.219698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 14:58:11.219698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 14:58:11.219698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:11.219698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 14:58:11.219698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:11.219698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:11.219698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 14:58:11.219698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 14:58:11.219698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 01 14:58:11.219698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return ref.to_dict() Aug 01 14:58:11.219698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 14:58:11.219698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application next(self.gen) Aug 01 14:58:11.220731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 14:58:11.220731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application yield resource Aug 01 14:58:11.220731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 14:58:11.220731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application next(self.gen) Aug 01 14:58:11.220731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 14:58:11.220731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 14:58:11.220731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 14:58:11.220731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 14:58:11.220731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 14:58:11.220731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise self.value Aug 01 14:58:11.220731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 14:58:11.220731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 14:58:11.220731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 14:58:11.220731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application session.commit() Aug 01 14:58:11.220731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 14:58:11.220731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 14:58:11.220731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 14:58:11.222555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 14:58:11.222555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 14:58:11.222555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self.session.flush() Aug 01 14:58:11.222555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 14:58:11.222555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 14:58:11.222555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 14:58:11.222555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 14:58:11.222555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 14:58:11.222555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application compat.raise_( Aug 01 14:58:11.222555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 14:58:11.222555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception Aug 01 14:58:11.222555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 14:58:11.222555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 14:58:11.222555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 14:58:11.222555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 14:58:11.222555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 14:58:11.222555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 14:58:11.224259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 14:58:11.224259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 14:58:11.224259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 14:58:11.224259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 14:58:11.224259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 14:58:11.224259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 14:58:11.224259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 14:58:11.224259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 14:58:11.224259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 14:58:11.224259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 14:58:11.224259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 14:58:11.224259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 14:58:11.224259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 14:58:11.224259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 14:58:11.224259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 14:58:11.224259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception Aug 01 14:58:11.225742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 14:58:11.225742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 14:58:11.225742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 14:58:11.225742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 14:58:11.225742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 14:58:11.225742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 14:58:11.225742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 14:58:11.225742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application conn.query(q) Aug 01 14:58:11.225742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 14:58:11.225742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 14:58:11.225742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 14:58:11.225742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application result.read() Aug 01 14:58:11.225742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 14:58:11.225742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 14:58:11.225742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 14:58:11.225742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 14:58:11.228197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 14:58:11.228197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 14:58:11.228197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 14:58:11.228197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 14:58:11.228197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 14:58:11.228197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 14:58:11.228197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [parameters: {'id': '6f3ad88546df4564b3f2a60e3af41d4d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 14:58:11.228197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 14:58:11.228197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:11.228197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:58:11.228197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:11.228197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:11.228197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:11.228197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:11.228197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:11.228197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:11.229221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:11.229221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:11.229221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:11.229221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:11.229221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:11.229221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:11.229221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 01 14:58:11.229221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 01 14:58:11.229221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:11.229221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:11.229221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 01 14:58:11.229221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 01 14:58:11.229221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 14:58:11.229221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 14:58:11.229221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 14:58:11.229221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:11.230163 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 232/478] 10.222.0.93 () {60 vars in 1131 bytes} [Mon Aug 1 14:58:10 2022] POST /identity/v3/roles => generated 144 bytes in 370 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:11.240491 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88c33a-7aa4-43c2-b0eb-bebede930c63 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:11.454876 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88c33a-7aa4-43c2-b0eb-bebede930c63 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:11.508983 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:11.529301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b9bff-b5c9-4775-890c-8e0fee15f366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:11.530548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b9bff-b5c9-4775-890c-8e0fee15f366 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:11.531244 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b9bff-b5c9-4775-890c-8e0fee15f366 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:11.531829 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b9bff-b5c9-4775-890c-8e0fee15f366 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:11.553393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b9bff-b5c9-4775-890c-8e0fee15f366 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:11.564087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-805b9bff-b5c9-4775-890c-8e0fee15f366 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:58:11.564087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:11.564087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:11.564087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:11.564087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:11.564087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:11.564087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:11.564087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:11.564087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:11.564087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:11.564087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:11.564087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:11.564087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:11.564087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:11.564087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:11.564087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:11.565581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:11.565581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:11.565581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:11.565581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:11.565581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:11.565581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:11.565581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:11.565581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:11.565581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:11.565581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:11.565581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:11.565581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:11.565581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:11.565581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:11.565581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:11.565581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:11.567044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:11.567044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:11.567044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:11.567044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:11.567044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:11.567044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:11.567044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:11.567044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:11.567044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:11.567044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:11.567044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:58:11.567044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:11.567044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 247/479] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:58:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:11.577009 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcc199-d332-4cfc-9ac7-b33583792280 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:11.682747 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:11.721481 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcc199-d332-4cfc-9ac7-b33583792280 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:11.766990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1925d-3240-49e6-9944-1f35d7351123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:11.767816 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1925d-3240-49e6-9944-1f35d7351123 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:11.768231 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1925d-3240-49e6-9944-1f35d7351123 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:11.768702 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1925d-3240-49e6-9944-1f35d7351123 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:11.769686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1925d-3240-49e6-9944-1f35d7351123 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:11.770044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1925d-3240-49e6-9944-1f35d7351123 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:11.772745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1925d-3240-49e6-9944-1f35d7351123 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:11.787131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 233/480] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:13.090939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b36b51-0122-48fc-96d4-b8af009e8fc7 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:13.091522 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b36b51-0122-48fc-96d4-b8af009e8fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:13.091882 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b36b51-0122-48fc-96d4-b8af009e8fc7 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:13.094773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 248/481] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:13.102424 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df073b2-87bc-4476-8039-6fa9d8d50b3a None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:13.103864 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df073b2-87bc-4476-8039-6fa9d8d50b3a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:13.104735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df073b2-87bc-4476-8039-6fa9d8d50b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:13.142827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-2df073b2-87bc-4476-8039-6fa9d8d50b3a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:13.144441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-2df073b2-87bc-4476-8039-6fa9d8d50b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:13.251348 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 234/482] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:13.294092 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b36b51-0122-48fc-96d4-b8af009e8fc7 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:13.442600 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b36b51-0122-48fc-96d4-b8af009e8fc7 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:13.483794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f940e8-d24f-41f1-9ba3-a2d52a0783a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:13.484922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f940e8-d24f-41f1-9ba3-a2d52a0783a8 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:13.485418 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f940e8-d24f-41f1-9ba3-a2d52a0783a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:13.485960 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f940e8-d24f-41f1-9ba3-a2d52a0783a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:13.512838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f940e8-d24f-41f1-9ba3-a2d52a0783a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:13.518922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-97f940e8-d24f-41f1-9ba3-a2d52a0783a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:13.518922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:13.518922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:13.518922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:13.518922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:13.518922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:13.518922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:13.518922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:13.518922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:13.518922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:13.518922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:13.518922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:13.518922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:13.518922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:13.518922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:13.518922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:13.519974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:13.519974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:13.519974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:13.519974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:13.519974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:13.519974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:13.519974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:13.519974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:13.519974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:13.519974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:13.519974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:13.519974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:13.519974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:13.519974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:13.519974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:13.519974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:13.521026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:13.521026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:13.521026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:13.521026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:13.521026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:13.521026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:13.521026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:13.521026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:13.521026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:13.521026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:13.521026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:13.521026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:13.524560 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 249/483] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:13.533408 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df073b2-87bc-4476-8039-6fa9d8d50b3a None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:13.613485 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:13.629445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af613e1d-0811-4a4b-b9b7-e9ee3591273e None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:13.629758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af613e1d-0811-4a4b-b9b7-e9ee3591273e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:13.630077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af613e1d-0811-4a4b-b9b7-e9ee3591273e None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:13.633083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 250/484] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:58:13.642467 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb29b2-ffa3-404f-a72e-448201548692 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:13.642858 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb29b2-ffa3-404f-a72e-448201548692 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:13.643030 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb29b2-ffa3-404f-a72e-448201548692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:13.681139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-4fdb29b2-ffa3-404f-a72e-448201548692 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:13.682365 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4fdb29b2-ffa3-404f-a72e-448201548692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:13.687450 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df073b2-87bc-4476-8039-6fa9d8d50b3a None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:13.764260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4217943-6d8c-4429-86ea-424b942c0fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:13.765949 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4217943-6d8c-4429-86ea-424b942c0fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:13.766773 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4217943-6d8c-4429-86ea-424b942c0fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:13.767563 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4217943-6d8c-4429-86ea-424b942c0fb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:13.769362 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4217943-6d8c-4429-86ea-424b942c0fb2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:13.770337 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4217943-6d8c-4429-86ea-424b942c0fb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:13.773521 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4217943-6d8c-4429-86ea-424b942c0fb2 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:13.790378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 235/485] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 14:58:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:13.793444 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:13.798093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4217943-6d8c-4429-86ea-424b942c0fb2 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:13.832046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 251/486] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:58:13.911914 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb29b2-ffa3-404f-a72e-448201548692 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:14.020116 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4217943-6d8c-4429-86ea-424b942c0fb2 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:14.079945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:14.095134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2a1e6-5258-4a6c-9d56-b6e57b6f7ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:14.096095 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2a1e6-5258-4a6c-9d56-b6e57b6f7ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:14.096581 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2a1e6-5258-4a6c-9d56-b6e57b6f7ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:14.097039 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2a1e6-5258-4a6c-9d56-b6e57b6f7ba8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:14.119646 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2a1e6-5258-4a6c-9d56-b6e57b6f7ba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:14.123974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2a1e6-5258-4a6c-9d56-b6e57b6f7ba8 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:14.154044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-84f2a1e6-5258-4a6c-9d56-b6e57b6f7ba8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:14.154044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:14.154044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:14.154044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:14.154044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:14.154044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:14.154044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:14.154044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:14.154044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:14.154044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:14.154044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:14.154044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:14.154044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:14.154044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:14.154044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:14.155848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:14.155848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:14.155848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:14.155848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:14.155848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:14.155848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:14.155848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:14.155848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:14.155848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:14.155848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:14.155848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:14.155848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:14.155848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:14.155848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:14.155848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:14.155848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:14.155848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:14.157592 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:14.157592 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:14.157592 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:14.157592 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:14.162387 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 236/487] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:14.166328 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f689847-0cbf-4cc7-9edb-8f76a530e736 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:14.167196 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f689847-0cbf-4cc7-9edb-8f76a530e736 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:14.167935 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f689847-0cbf-4cc7-9edb-8f76a530e736 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:14.169901 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb29b2-ffa3-404f-a72e-448201548692 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:14.173191 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 237/488] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:14 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:14.175196 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f689847-0cbf-4cc7-9edb-8f76a530e736 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:14.261310 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c16831-24f2-4f9a-b24c-c0660c19f4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:14.261989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c16831-24f2-4f9a-b24c-c0660c19f4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:14.262300 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c16831-24f2-4f9a-b24c-c0660c19f4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:14.262560 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c16831-24f2-4f9a-b24c-c0660c19f4a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:14.276606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c16831-24f2-4f9a-b24c-c0660c19f4a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:14.282497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-01c16831-24f2-4f9a-b24c-c0660c19f4a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:14.282497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:14.282497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:14.282497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:14.282497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:14.282497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:14.282497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:14.282497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:14.282497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:14.282497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:14.282497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:14.282497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:14.282497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:14.282497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:14.282497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:14.282497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:14.284298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:14.284298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:14.284298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:14.284298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:14.284298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:14.284298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:14.284298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:14.284298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:14.284298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:14.284298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:14.284298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:14.284298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:14.284298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:14.284298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:14.284298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:14.284298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:14.285880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:14.285880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:14.285880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:14.285880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:14.285880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:14.285880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:14.285880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:14.285880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:14.285880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:14.285880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:14.285880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:14.285880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:14.289295 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 252/489] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:14.291339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05f12c-5af7-44c6-8ccf-db1057173142 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:14.291339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05f12c-5af7-44c6-8ccf-db1057173142 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:14.291659 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05f12c-5af7-44c6-8ccf-db1057173142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:14.312262 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-9f05f12c-5af7-44c6-8ccf-db1057173142 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:14.313772 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-9f05f12c-5af7-44c6-8ccf-db1057173142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:14.348613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:14.361537 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f689847-0cbf-4cc7-9edb-8f76a530e736 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:14.408638 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 253/490] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:14.409640 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05f12c-5af7-44c6-8ccf-db1057173142 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:14.431904 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd83172-9b32-4d99-8533-c35026cc5e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:14.433460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd83172-9b32-4d99-8533-c35026cc5e39 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:14.434314 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd83172-9b32-4d99-8533-c35026cc5e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:14.435095 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd83172-9b32-4d99-8533-c35026cc5e39 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:14.436850 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd83172-9b32-4d99-8533-c35026cc5e39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:14.437985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd83172-9b32-4d99-8533-c35026cc5e39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:14.442324 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd83172-9b32-4d99-8533-c35026cc5e39 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:14.467457 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 238/491] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:14.469529 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd83172-9b32-4d99-8533-c35026cc5e39 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:14.547246 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:14.569801 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05f12c-5af7-44c6-8ccf-db1057173142 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:14.625751 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfed335-c20f-4437-af94-9a0fd51f42f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:14.626454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfed335-c20f-4437-af94-9a0fd51f42f9 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:14.626817 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfed335-c20f-4437-af94-9a0fd51f42f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:14.627094 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfed335-c20f-4437-af94-9a0fd51f42f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:14.627828 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfed335-c20f-4437-af94-9a0fd51f42f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:14.628767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfed335-c20f-4437-af94-9a0fd51f42f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:14.632115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfed335-c20f-4437-af94-9a0fd51f42f9 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:14.655944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 254/492] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 14:58:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:14.655944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfed335-c20f-4437-af94-9a0fd51f42f9 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:14.660303 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd83172-9b32-4d99-8533-c35026cc5e39 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:14.709453 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:14.724569 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1142a5-4caf-4963-b223-50ffcb8e029b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:14.725553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1142a5-4caf-4963-b223-50ffcb8e029b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:14.726000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1142a5-4caf-4963-b223-50ffcb8e029b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:14.726444 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1142a5-4caf-4963-b223-50ffcb8e029b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:14.745266 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1142a5-4caf-4963-b223-50ffcb8e029b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:14.755474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-0c1142a5-4caf-4963-b223-50ffcb8e029b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:14.755474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:14.755474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:14.755474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:14.755474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:14.755474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:14.755474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:14.755474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:14.755474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:14.755474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:14.755474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:14.755474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:14.755474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:14.755474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:14.755474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:14.755474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:14.758236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:14.758236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:14.758236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:14.758236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:14.758236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:14.758236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:14.758236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:14.758236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:14.758236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:14.758236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:14.758236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:14.758236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:14.758236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:14.758236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:14.758236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:14.758236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:14.759796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:14.759796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:14.759796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:14.759796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:14.759796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:14.759796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:14.759796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:14.759796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:14.759796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:14.759796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:14.759796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:14.759796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:14.759796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 239/493] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:14.762891 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f80973-7e60-4490-8495-4f5eb7f8aa67 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:14.762891 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f80973-7e60-4490-8495-4f5eb7f8aa67 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:14.762891 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f80973-7e60-4490-8495-4f5eb7f8aa67 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:14.764170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 240/494] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:14.769241 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f80973-7e60-4490-8495-4f5eb7f8aa67 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:14.862804 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfed335-c20f-4437-af94-9a0fd51f42f9 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:14.915359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae1767-b8cb-44bd-8d40-1a18934a6679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:14.916320 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae1767-b8cb-44bd-8d40-1a18934a6679 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:14.916802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae1767-b8cb-44bd-8d40-1a18934a6679 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:14.917257 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae1767-b8cb-44bd-8d40-1a18934a6679 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:14.928848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae1767-b8cb-44bd-8d40-1a18934a6679 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:14.930912 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae1767-b8cb-44bd-8d40-1a18934a6679 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:14.950204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-09ae1767-b8cb-44bd-8d40-1a18934a6679 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 14:58:14.950204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:14.950204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:14.950204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:14.950204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:14.950204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:14.950204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:14.950204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:14.950204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:14.950204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:14.950204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:14.950204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:14.950204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:14.950204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:14.950204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:14.950204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:14.956839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:14.956839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:14.956839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:14.956839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:14.956839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:14.956839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:14.956839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:14.956839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:14.956839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:14.956839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:14.956839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:14.956839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:14.956839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:14.956839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:14.956839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:14.956839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:14.956839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:14.956839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:14.958989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:14.958989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:14.958989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:14.958989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:14.958989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:14.958989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:14.958989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:14.958989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:14.958989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 14:58:14.958989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:14.958989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 255/495] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:58:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:14.958989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae1767-b8cb-44bd-8d40-1a18934a6679 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:14.964140 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:14.972159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f80973-7e60-4490-8495-4f5eb7f8aa67 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:15.031806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f1dab-f3b2-48c9-83dc-53edb8d6b18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:15.032713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f1dab-f3b2-48c9-83dc-53edb8d6b18c None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:15.033076 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f1dab-f3b2-48c9-83dc-53edb8d6b18c None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:15.033432 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f1dab-f3b2-48c9-83dc-53edb8d6b18c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:15.051016 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f1dab-f3b2-48c9-83dc-53edb8d6b18c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:15.053597 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f1dab-f3b2-48c9-83dc-53edb8d6b18c None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:15.082325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-5d6f1dab-f3b2-48c9-83dc-53edb8d6b18c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:15.082325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:15.082325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:15.082325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:15.082325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:15.082325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:15.082325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:15.082325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:15.082325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:15.082325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:15.082325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:15.082325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:15.082325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:15.082325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:15.082325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:15.084683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:15.084683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:15.084683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:15.084683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:15.084683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:15.084683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:15.084683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:15.084683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:15.084683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:15.084683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:15.084683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:15.084683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:15.084683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:15.084683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:15.084683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:15.084683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:15.084683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:15.086630 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:15.086630 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:15.086630 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:15.086630 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:15.090041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 241/496] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:15.092813 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83712b-c2bb-4467-b948-82a8ca20de94 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:15.093582 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83712b-c2bb-4467-b948-82a8ca20de94 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:15.094412 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83712b-c2bb-4467-b948-82a8ca20de94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:15.115848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:15.129055 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-9a83712b-c2bb-4467-b948-82a8ca20de94 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:15.131749 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-9a83712b-c2bb-4467-b948-82a8ca20de94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:15.138174 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae1767-b8cb-44bd-8d40-1a18934a6679 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:15.194874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea64919-a233-44b1-8cd9-fade52b1208c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:15.196668 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea64919-a233-44b1-8cd9-fade52b1208c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:15.197170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea64919-a233-44b1-8cd9-fade52b1208c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:15.197663 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea64919-a233-44b1-8cd9-fade52b1208c None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:15.199034 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea64919-a233-44b1-8cd9-fade52b1208c None admin] RBAC: Adding query filter params (name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:15.199563 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea64919-a233-44b1-8cd9-fade52b1208c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:15.202275 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea64919-a233-44b1-8cd9-fade52b1208c None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:15.223280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 256/497] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 14:58:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:15.223280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea64919-a233-44b1-8cd9-fade52b1208c None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:15.276499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 242/498] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:15.276499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83712b-c2bb-4467-b948-82a8ca20de94 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:15.416709 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:15.430265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea64919-a233-44b1-8cd9-fade52b1208c None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:15.460866 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83712b-c2bb-4467-b948-82a8ca20de94 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:15.507981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4690f86-4fea-4737-bd5a-1f5b58240179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:15.509177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4690f86-4fea-4737-bd5a-1f5b58240179 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:15.509621 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4690f86-4fea-4737-bd5a-1f5b58240179 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:15.510119 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4690f86-4fea-4737-bd5a-1f5b58240179 None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:15.511669 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4690f86-4fea-4737-bd5a-1f5b58240179 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:15.512250 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4690f86-4fea-4737-bd5a-1f5b58240179 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:15.515231 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4690f86-4fea-4737-bd5a-1f5b58240179 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:15.530208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2d426-c4c6-4a3e-b696-078f4f28a436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:15.531263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2d426-c4c6-4a3e-b696-078f4f28a436 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:15.531769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2d426-c4c6-4a3e-b696-078f4f28a436 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:15.532264 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2d426-c4c6-4a3e-b696-078f4f28a436 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:15.533647 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2d426-c4c6-4a3e-b696-078f4f28a436 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:15.534144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2d426-c4c6-4a3e-b696-078f4f28a436 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:15.536866 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2d426-c4c6-4a3e-b696-078f4f28a436 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:15.560519 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 243/499] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:15.562546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2d426-c4c6-4a3e-b696-078f4f28a436 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:15.591784 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 257/500] 10.222.0.93 () {56 vars in 1137 bytes} [Mon Aug 1 14:58:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:15.593315 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4690f86-4fea-4737-bd5a-1f5b58240179 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:15.607821 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:15.718080 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4690f86-4fea-4737-bd5a-1f5b58240179 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:15.741702 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2d426-c4c6-4a3e-b696-078f4f28a436 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:15.758198 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54a199-5fa8-4b19-8a87-906dc85286c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:15.758702 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54a199-5fa8-4b19-8a87-906dc85286c2 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:15.759006 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54a199-5fa8-4b19-8a87-906dc85286c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:15.759282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54a199-5fa8-4b19-8a87-906dc85286c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:15.773631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54a199-5fa8-4b19-8a87-906dc85286c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:15.781896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-1b54a199-5fa8-4b19-8a87-906dc85286c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:15.781896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:15.781896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:15.781896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:15.781896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:15.781896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:15.781896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:15.781896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:15.781896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:15.781896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:15.781896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:15.781896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:15.781896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:15.781896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:15.781896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:15.781896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:15.783338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:15.783338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:15.783338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:15.783338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:15.783338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:15.783338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:15.783338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:15.783338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:15.783338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:15.783338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:15.783338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:15.783338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:15.783338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:15.783338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:15.783338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:15.783338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:15.789922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:15.789922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:15.789922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:15.789922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:15.789922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:15.789922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:15.789922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:15.789922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:15.789922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:15.789922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:15.789922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:15.789922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:15.789922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 258/501] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:15.789922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54a199-5fa8-4b19-8a87-906dc85286c2 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:15.808449 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395d1df-37bb-4b06-9ad2-d2e5911c529a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:15.809553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395d1df-37bb-4b06-9ad2-d2e5911c529a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:15.810022 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395d1df-37bb-4b06-9ad2-d2e5911c529a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:15.810527 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395d1df-37bb-4b06-9ad2-d2e5911c529a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:15.828274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:15.832803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395d1df-37bb-4b06-9ad2-d2e5911c529a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:15.837078 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395d1df-37bb-4b06-9ad2-d2e5911c529a None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:15.862505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-4395d1df-37bb-4b06-9ad2-d2e5911c529a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:15.862505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:15.862505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:15.862505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:15.862505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:15.862505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:15.862505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:15.862505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:15.862505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:15.862505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:15.862505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:15.862505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:15.862505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:15.862505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:15.862505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:15.864144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:15.864144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:15.864144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:15.864144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:15.864144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:15.864144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:15.864144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:15.864144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:15.864144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:15.864144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:15.864144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:15.864144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:15.864144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:15.864144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:15.864144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:15.864144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:15.864144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:15.865382 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:15.865382 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:15.865382 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:15.865382 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:15.865382 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 244/502] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:15.866485 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe559b-66f6-4b6d-b14f-7d3095a316e3 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:15.866967 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe559b-66f6-4b6d-b14f-7d3095a316e3 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:15.867428 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe559b-66f6-4b6d-b14f-7d3095a316e3 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:15.868089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:15.870074 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 245/503] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:15.872159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe559b-66f6-4b6d-b14f-7d3095a316e3 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:15.916386 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54a199-5fa8-4b19-8a87-906dc85286c2 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:15.961669 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d930965-7062-4fb0-9943-9c1eeb1130ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:15.962486 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d930965-7062-4fb0-9943-9c1eeb1130ae None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:15.962870 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d930965-7062-4fb0-9943-9c1eeb1130ae None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:15.963396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d930965-7062-4fb0-9943-9c1eeb1130ae None admin] PATH_INFO: `/v3/users/nova` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:15.974562 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d930965-7062-4fb0-9943-9c1eeb1130ae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:15.976208 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d930965-7062-4fb0-9943-9c1eeb1130ae None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:15.990503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-0d930965-7062-4fb0-9943-9c1eeb1130ae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:58:15.990503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:15.990503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:15.990503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:15.990503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:15.990503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:15.990503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:15.990503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:15.990503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:15.990503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:15.990503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:15.990503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:15.990503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:15.990503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:15.990503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:15.990503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:15.992404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:15.992404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:15.992404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:15.992404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:15.992404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:15.992404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:15.992404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:15.992404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:15.992404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:15.992404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:15.992404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:15.992404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:15.992404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:15.992404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:15.992404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:15.992404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:15.992404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:15.992404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:15.994182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:15.994182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:15.994182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:15.994182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:15.994182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:15.994182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:15.994182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:15.994182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:15.994182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:58:15.994182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:15.994182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 259/504] 10.222.0.93 () {56 vars in 1093 bytes} [Mon Aug 1 14:58:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:15.994182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d930965-7062-4fb0-9943-9c1eeb1130ae None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:16.012273 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe559b-66f6-4b6d-b14f-7d3095a316e3 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:16.065077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffea758-a0d9-45a1-8ee5-e7744d8d3444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:16.065795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffea758-a0d9-45a1-8ee5-e7744d8d3444 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:16.066204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffea758-a0d9-45a1-8ee5-e7744d8d3444 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:16.066600 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffea758-a0d9-45a1-8ee5-e7744d8d3444 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:16.167313 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:16.172512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d930965-7062-4fb0-9943-9c1eeb1130ae None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:16.179035 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffea758-a0d9-45a1-8ee5-e7744d8d3444 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:16.180575 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffea758-a0d9-45a1-8ee5-e7744d8d3444 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:16.220104 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:16.235349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4026e-f497-4d13-a28f-8818671734e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:16.236281 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4026e-f497-4d13-a28f-8818671734e3 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:16.236796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4026e-f497-4d13-a28f-8818671734e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:16.237250 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4026e-f497-4d13-a28f-8818671734e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:16.238477 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4026e-f497-4d13-a28f-8818671734e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:16.239099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4026e-f497-4d13-a28f-8818671734e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:16.241444 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4026e-f497-4d13-a28f-8818671734e3 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:16.253602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 260/505] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 14:58:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:16.254501 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4026e-f497-4d13-a28f-8818671734e3 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:16.316405 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 246/506] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:58:16.317956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2023810-7b3f-4420-87cc-a4d6287ce975 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:16.318949 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2023810-7b3f-4420-87cc-a4d6287ce975 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:16.319357 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2023810-7b3f-4420-87cc-a4d6287ce975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:16.341553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-c2023810-7b3f-4420-87cc-a4d6287ce975 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:16.343105 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c2023810-7b3f-4420-87cc-a4d6287ce975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:16.403575 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4026e-f497-4d13-a28f-8818671734e3 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:16.428863 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 247/507] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:16.430220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2023810-7b3f-4420-87cc-a4d6287ce975 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:16.469510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299be89-0bbe-4228-be1c-de449f75d2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:16.470134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299be89-0bbe-4228-be1c-de449f75d2df None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:16.470411 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299be89-0bbe-4228-be1c-de449f75d2df None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:16.470717 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299be89-0bbe-4228-be1c-de449f75d2df None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:16.471581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299be89-0bbe-4228-be1c-de449f75d2df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:16.472039 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299be89-0bbe-4228-be1c-de449f75d2df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:16.473746 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299be89-0bbe-4228-be1c-de449f75d2df None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:16.482356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:16.490783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 261/508] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:16.493875 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299be89-0bbe-4228-be1c-de449f75d2df None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:16.576339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:16.602461 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2023810-7b3f-4420-87cc-a4d6287ce975 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:16.644266 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e26a8-998e-478b-93bb-d6db2d18ee5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:16.644912 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e26a8-998e-478b-93bb-d6db2d18ee5c None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:16.645326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e26a8-998e-478b-93bb-d6db2d18ee5c None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:16.645616 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e26a8-998e-478b-93bb-d6db2d18ee5c None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:16.646583 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e26a8-998e-478b-93bb-d6db2d18ee5c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:16.646900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e26a8-998e-478b-93bb-d6db2d18ee5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:16.648579 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e26a8-998e-478b-93bb-d6db2d18ee5c None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:16.681868 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299be89-0bbe-4228-be1c-de449f75d2df None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:16.697643 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 248/509] 10.222.0.93 () {56 vars in 1133 bytes} [Mon Aug 1 14:58:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:16.698293 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e26a8-998e-478b-93bb-d6db2d18ee5c None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:16.732992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd86d70-4910-446d-850e-4f8d5cf0a844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:16.733793 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd86d70-4910-446d-850e-4f8d5cf0a844 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:16.734217 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd86d70-4910-446d-850e-4f8d5cf0a844 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:16.734670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd86d70-4910-446d-850e-4f8d5cf0a844 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:16.748454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd86d70-4910-446d-850e-4f8d5cf0a844 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:16.750069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd86d70-4910-446d-850e-4f8d5cf0a844 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:16.769057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-5bd86d70-4910-446d-850e-4f8d5cf0a844 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:16.769057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:16.769057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:16.769057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:16.769057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:16.769057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:16.769057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:16.769057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:16.769057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:16.769057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:16.769057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:16.769057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:16.769057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:16.769057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:16.769057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:16.770961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:16.770961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:16.770961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:16.770961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:16.770961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:16.770961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:16.770961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:16.770961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:16.770961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:16.770961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:16.770961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:16.770961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:16.770961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:16.770961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:16.770961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:16.770961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:16.770961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:16.773771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:16.773771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:16.773771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:16.773771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:16.773771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 262/510] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:16.773771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd86d70-4910-446d-850e-4f8d5cf0a844 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:16.826760 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:16.862806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e26a8-998e-478b-93bb-d6db2d18ee5c None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:16.894683 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:16.899281 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd86d70-4910-446d-850e-4f8d5cf0a844 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:16.923980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d71dce7-ae12-4b89-b9f7-908d349e3951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:16.924842 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71dce7-ae12-4b89-b9f7-908d349e3951 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:16.925206 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71dce7-ae12-4b89-b9f7-908d349e3951 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:16.925638 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71dce7-ae12-4b89-b9f7-908d349e3951 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:16.942327 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d71dce7-ae12-4b89-b9f7-908d349e3951 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:16.945171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d71dce7-ae12-4b89-b9f7-908d349e3951 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:16.945453 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ab0f4-0f98-48bc-9f15-49d9bf262509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:16.946170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ab0f4-0f98-48bc-9f15-49d9bf262509 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:16.946444 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ab0f4-0f98-48bc-9f15-49d9bf262509 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:16.946722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ab0f4-0f98-48bc-9f15-49d9bf262509 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:16.963592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ab0f4-0f98-48bc-9f15-49d9bf262509 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:16.965327 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ab0f4-0f98-48bc-9f15-49d9bf262509 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:16.967483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-9d71dce7-ae12-4b89-b9f7-908d349e3951 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:16.967483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:16.967483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:16.967483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:16.967483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:16.967483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:16.967483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:16.967483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:16.967483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:16.967483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:16.967483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:16.967483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:16.967483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:16.967483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:16.967483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:16.969189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:16.969189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:16.969189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:16.969189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:16.969189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:16.969189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:16.969189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:16.969189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:16.969189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:16.969189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:16.969189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:16.969189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:16.969189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:16.969189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:16.969189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:16.969189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:16.969189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:16.970886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:16.970886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:16.970886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:16.970886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:16.970886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 249/511] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:16.970886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d71dce7-ae12-4b89-b9f7-908d349e3951 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:17.000159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-758ab0f4-0f98-48bc-9f15-49d9bf262509 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:17.000159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:17.000159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:17.000159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:17.000159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:17.000159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:17.000159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:17.000159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:17.000159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:17.000159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:17.000159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:17.000159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:17.000159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:17.000159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:17.000159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:17.001655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:17.001655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:17.001655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:17.001655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:17.001655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:17.001655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:17.001655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:17.001655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:17.001655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:17.001655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:17.001655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:17.001655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:17.001655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:17.001655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:17.001655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:17.001655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:17.001655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:17.002978 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:17.002978 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:17.002978 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:17.002978 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:17.002978 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 263/512] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:17.004037 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ab0f4-0f98-48bc-9f15-49d9bf262509 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:17.125546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d71dce7-ae12-4b89-b9f7-908d349e3951 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:17.179029 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:17.182041 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ab0f4-0f98-48bc-9f15-49d9bf262509 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:17.186965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736d71a-2e83-4b7b-8d3f-2639143a95c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:17.187995 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736d71a-2e83-4b7b-8d3f-2639143a95c8 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:17.188426 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736d71a-2e83-4b7b-8d3f-2639143a95c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:17.189066 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736d71a-2e83-4b7b-8d3f-2639143a95c8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:17.199767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736d71a-2e83-4b7b-8d3f-2639143a95c8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:17.201729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736d71a-2e83-4b7b-8d3f-2639143a95c8 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:17.219396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-6736d71a-2e83-4b7b-8d3f-2639143a95c8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 14:58:17.219396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:17.219396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:17.219396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:17.219396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:17.219396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:17.219396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:17.219396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:17.219396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:17.219396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:17.219396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:17.219396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:17.219396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:17.219396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:17.219396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:17.219396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:17.221204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:17.221204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:17.221204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:17.221204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:17.221204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:17.221204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:17.221204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:17.221204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:17.221204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:17.221204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:17.221204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:17.221204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:17.221204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:17.221204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:17.221204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:17.221204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:17.221204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:17.221204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:17.223661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:17.223661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:17.223661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:17.223661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:17.223661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:17.223661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:17.223661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:17.223661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:17.223661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 14:58:17.223661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:17.223661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 250/513] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:58:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:17.223661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736d71a-2e83-4b7b-8d3f-2639143a95c8 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:17.245581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfea84-5242-473d-9c0b-006f2522a127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:17.246853 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfea84-5242-473d-9c0b-006f2522a127 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:17.247330 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfea84-5242-473d-9c0b-006f2522a127 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:17.247799 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfea84-5242-473d-9c0b-006f2522a127 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:17.277856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:17.350014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfea84-5242-473d-9c0b-006f2522a127 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:17.351503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfea84-5242-473d-9c0b-006f2522a127 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:17.367053 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736d71a-2e83-4b7b-8d3f-2639143a95c8 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:17.414954 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a6b0a-ea2d-47ff-a282-31b09df39b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:17.415586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a6b0a-ea2d-47ff-a282-31b09df39b63 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:17.415897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a6b0a-ea2d-47ff-a282-31b09df39b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:17.416175 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a6b0a-ea2d-47ff-a282-31b09df39b63 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:17.416983 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a6b0a-ea2d-47ff-a282-31b09df39b63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:17.417288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a6b0a-ea2d-47ff-a282-31b09df39b63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:17.418963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a6b0a-ea2d-47ff-a282-31b09df39b63 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:17.434040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 251/514] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:17.434948 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a6b0a-ea2d-47ff-a282-31b09df39b63 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:17.487498 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 264/515] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:58:17.488628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfea84-5242-473d-9c0b-006f2522a127 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:17.525551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:17.588923 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:17.589928 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a6b0a-ea2d-47ff-a282-31b09df39b63 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:17.640406 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd2559-0a2b-4d1f-a1d2-be82c581ce5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:17.641330 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd2559-0a2b-4d1f-a1d2-be82c581ce5f None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:17.641791 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd2559-0a2b-4d1f-a1d2-be82c581ce5f None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:17.642252 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd2559-0a2b-4d1f-a1d2-be82c581ce5f None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:17.643426 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd2559-0a2b-4d1f-a1d2-be82c581ce5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:17.643917 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd2559-0a2b-4d1f-a1d2-be82c581ce5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:17.646793 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd2559-0a2b-4d1f-a1d2-be82c581ce5f None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:17.654978 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfea84-5242-473d-9c0b-006f2522a127 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:17.664735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 252/516] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:17.666041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd2559-0a2b-4d1f-a1d2-be82c581ce5f None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:17.701813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaabd80-d52a-423b-b0a4-fab18f6f9d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:17.702635 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaabd80-d52a-423b-b0a4-fab18f6f9d6c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:17.702954 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaabd80-d52a-423b-b0a4-fab18f6f9d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:17.703287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaabd80-d52a-423b-b0a4-fab18f6f9d6c None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:17.704300 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaabd80-d52a-423b-b0a4-fab18f6f9d6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:17.704801 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaabd80-d52a-423b-b0a4-fab18f6f9d6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:17.706595 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaabd80-d52a-423b-b0a4-fab18f6f9d6c None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:17.722535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 265/517] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:17.723616 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaabd80-d52a-423b-b0a4-fab18f6f9d6c None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:17.814901 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd2559-0a2b-4d1f-a1d2-be82c581ce5f None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:17.832051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:17.857635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d047c49-fd9e-49e3-86f6-90dd7a13f37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:17.859088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d047c49-fd9e-49e3-86f6-90dd7a13f37b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:17.859497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d047c49-fd9e-49e3-86f6-90dd7a13f37b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:17.859945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d047c49-fd9e-49e3-86f6-90dd7a13f37b None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:17.860987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d047c49-fd9e-49e3-86f6-90dd7a13f37b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:17.861417 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d047c49-fd9e-49e3-86f6-90dd7a13f37b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:17.861861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaabd80-d52a-423b-b0a4-fab18f6f9d6c None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:17.863294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d047c49-fd9e-49e3-86f6-90dd7a13f37b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:17.879241 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:17.887357 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 253/518] 10.222.0.93 () {56 vars in 1137 bytes} [Mon Aug 1 14:58:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:17.888434 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d047c49-fd9e-49e3-86f6-90dd7a13f37b None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:17.907059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541ae43-4856-431b-b139-74140096612d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:17.907744 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541ae43-4856-431b-b139-74140096612d None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:17.908045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541ae43-4856-431b-b139-74140096612d None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:17.908375 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541ae43-4856-431b-b139-74140096612d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:17.918455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541ae43-4856-431b-b139-74140096612d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:17.920038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541ae43-4856-431b-b139-74140096612d None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:17.933890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-8541ae43-4856-431b-b139-74140096612d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 14:58:17.933890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:17.933890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:17.933890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:17.933890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:17.933890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:17.933890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:17.933890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:17.933890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:17.933890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:17.933890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:17.933890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:17.933890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:17.933890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:17.933890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:17.933890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:17.935424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:17.935424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:17.935424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:17.935424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:17.935424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:17.935424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:17.935424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:17.935424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:17.935424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:17.935424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:17.935424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:17.935424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:17.935424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:17.935424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:17.935424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:17.935424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:17.935424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:17.935424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:17.936716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:17.936716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:17.936716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:17.936716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:17.936716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:17.936716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:17.936716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:17.936716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:17.936716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 14:58:17.936716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:17.936716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 266/519] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:17.936716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541ae43-4856-431b-b139-74140096612d None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:18.044666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d047c49-fd9e-49e3-86f6-90dd7a13f37b None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:18.097222 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27d727-44d8-4f29-9250-86d15b5f3676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:18.097908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27d727-44d8-4f29-9250-86d15b5f3676 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:18.098176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27d727-44d8-4f29-9250-86d15b5f3676 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:18.098454 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27d727-44d8-4f29-9250-86d15b5f3676 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:18.100436 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541ae43-4856-431b-b139-74140096612d None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:18.110700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27d727-44d8-4f29-9250-86d15b5f3676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:18.112093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27d727-44d8-4f29-9250-86d15b5f3676 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:18.129942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-bf27d727-44d8-4f29-9250-86d15b5f3676 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:18.129942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:18.129942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:18.129942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:18.129942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:18.129942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:18.129942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:18.129942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:18.129942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:18.129942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:18.129942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:18.129942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:18.129942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:18.129942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:18.129942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:18.131553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:18.131553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:18.131553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:18.131553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:18.131553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:18.131553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:18.131553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:18.131553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:18.131553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:18.131553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:18.131553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:18.131553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:18.131553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:18.131553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:18.131553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:18.131553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:18.131553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:18.132526 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:18.132526 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:18.132526 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:18.132526 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:18.132526 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 254/520] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:18.132816 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27d727-44d8-4f29-9250-86d15b5f3676 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:18.162608 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fea12-03fd-47a6-a3a6-a1f336c4a438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:18.163711 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fea12-03fd-47a6-a3a6-a1f336c4a438 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:18.164196 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fea12-03fd-47a6-a3a6-a1f336c4a438 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:18.164753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fea12-03fd-47a6-a3a6-a1f336c4a438 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:18.169121 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:18.178244 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fea12-03fd-47a6-a3a6-a1f336c4a438 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:18.181222 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fea12-03fd-47a6-a3a6-a1f336c4a438 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:18.205361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-329fea12-03fd-47a6-a3a6-a1f336c4a438 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:18.205361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:18.205361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:18.205361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:18.205361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:18.205361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:18.205361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:18.205361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:18.205361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:18.205361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:18.205361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:18.205361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:18.205361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:18.205361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:18.205361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:18.206503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:18.206503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:18.206503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:18.206503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:18.206503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:18.206503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:18.206503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:18.206503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:18.206503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:18.206503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:18.206503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:18.206503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:18.206503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:18.206503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:18.206503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:18.206503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:18.206503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:18.207677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:18.207677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:18.207677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:18.207677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:18.207677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:18.207677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:18.207677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:18.207677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:18.207677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 267/521] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:18.209186 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fea12-03fd-47a6-a3a6-a1f336c4a438 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:18.267239 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:18.314647 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27d727-44d8-4f29-9250-86d15b5f3676 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:18.366043 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e112f-b3bd-4a66-b9b5-2ba3bc2f63d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:18.366791 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e112f-b3bd-4a66-b9b5-2ba3bc2f63d2 None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:18.367211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e112f-b3bd-4a66-b9b5-2ba3bc2f63d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:18.367544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e112f-b3bd-4a66-b9b5-2ba3bc2f63d2 None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:18.368299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e112f-b3bd-4a66-b9b5-2ba3bc2f63d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:18.369969 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e112f-b3bd-4a66-b9b5-2ba3bc2f63d2 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:18.370634 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fea12-03fd-47a6-a3a6-a1f336c4a438 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:18.399396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 255/522] 10.222.0.93 () {60 vars in 1132 bytes} [Mon Aug 1 14:58:18 2022] POST /identity/v3/users => generated 247 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:58:18.400282 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e112f-b3bd-4a66-b9b5-2ba3bc2f63d2 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:18.439963 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b459c-95d8-4fd4-bc9e-8ee92a10be9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:18.440635 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b459c-95d8-4fd4-bc9e-8ee92a10be9d None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:18.440918 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b459c-95d8-4fd4-bc9e-8ee92a10be9d None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:18.441189 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b459c-95d8-4fd4-bc9e-8ee92a10be9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:18.555796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b459c-95d8-4fd4-bc9e-8ee92a10be9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:18.557421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b459c-95d8-4fd4-bc9e-8ee92a10be9d None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:18.572518 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:18.612283 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e112f-b3bd-4a66-b9b5-2ba3bc2f63d2 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:18.619477 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:18.667024 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d9389-1b04-474f-ba15-6b5e3ebdcd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:18.668025 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d9389-1b04-474f-ba15-6b5e3ebdcd2b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:18.668533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d9389-1b04-474f-ba15-6b5e3ebdcd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:18.669001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d9389-1b04-474f-ba15-6b5e3ebdcd2b None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:18.670352 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d9389-1b04-474f-ba15-6b5e3ebdcd2b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:18.670894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d9389-1b04-474f-ba15-6b5e3ebdcd2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:18.673645 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d9389-1b04-474f-ba15-6b5e3ebdcd2b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:18.707912 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 256/523] 10.222.0.93 () {56 vars in 1139 bytes} [Mon Aug 1 14:58:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:18.712622 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d9389-1b04-474f-ba15-6b5e3ebdcd2b None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:18.744929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 268/524] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:58:18.750097 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b459c-95d8-4fd4-bc9e-8ee92a10be9d None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:18.865364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d9389-1b04-474f-ba15-6b5e3ebdcd2b None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:18.930636 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a2339-c3de-4259-92ff-376cd1c28e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:18.932029 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a2339-c3de-4259-92ff-376cd1c28e40 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:18.932646 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a2339-c3de-4259-92ff-376cd1c28e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:18.933138 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a2339-c3de-4259-92ff-376cd1c28e40 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:18.934394 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a2339-c3de-4259-92ff-376cd1c28e40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:18.934896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a2339-c3de-4259-92ff-376cd1c28e40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:18.937655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a2339-c3de-4259-92ff-376cd1c28e40 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:18.941588 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b459c-95d8-4fd4-bc9e-8ee92a10be9d None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:18.942360 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:18.963765 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 257/525] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:18.965644 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a2339-c3de-4259-92ff-376cd1c28e40 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:19.001440 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:19.010981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e935a48-f392-4a8f-abeb-1048f9e5607e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:19.012121 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e935a48-f392-4a8f-abeb-1048f9e5607e None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:19.012732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e935a48-f392-4a8f-abeb-1048f9e5607e None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:19.013284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e935a48-f392-4a8f-abeb-1048f9e5607e None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:19.014784 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e935a48-f392-4a8f-abeb-1048f9e5607e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:19.015458 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e935a48-f392-4a8f-abeb-1048f9e5607e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:19.018131 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e935a48-f392-4a8f-abeb-1048f9e5607e None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:19.060514 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 269/526] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:19.060514 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e935a48-f392-4a8f-abeb-1048f9e5607e None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:19.150767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a2339-c3de-4259-92ff-376cd1c28e40 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:19.206060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abb31f-750f-44ed-9cf6-bb39a2801a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:19.206859 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abb31f-750f-44ed-9cf6-bb39a2801a72 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:19.207308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abb31f-750f-44ed-9cf6-bb39a2801a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:19.207769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abb31f-750f-44ed-9cf6-bb39a2801a72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:19.250507 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e935a48-f392-4a8f-abeb-1048f9e5607e None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:19.295751 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:19.303631 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abb31f-750f-44ed-9cf6-bb39a2801a72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:19.305257 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abb31f-750f-44ed-9cf6-bb39a2801a72 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:19.317194 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa26da5-7a4e-4e39-bbed-69fae3975da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:19.319211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa26da5-7a4e-4e39-bbed-69fae3975da7 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:19.319676 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa26da5-7a4e-4e39-bbed-69fae3975da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:19.320157 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa26da5-7a4e-4e39-bbed-69fae3975da7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:19.340070 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa26da5-7a4e-4e39-bbed-69fae3975da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:19.343012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa26da5-7a4e-4e39-bbed-69fae3975da7 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:19.373932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-5fa26da5-7a4e-4e39-bbed-69fae3975da7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:19.373932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:19.373932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:19.373932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:19.373932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:19.373932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:19.373932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:19.373932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:19.373932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:19.373932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:19.373932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:19.373932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:19.373932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:19.373932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:19.373932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:19.375338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:19.375338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:19.375338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:19.375338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:19.375338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:19.375338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:19.375338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:19.375338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:19.375338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:19.375338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:19.375338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:19.375338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:19.375338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:19.375338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:19.375338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:19.375338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:19.375338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:19.376432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:19.376432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:19.376432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:19.376432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:19.377761 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 270/527] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:19.378057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa26da5-7a4e-4e39-bbed-69fae3975da7 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:19.458356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 258/528] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:58:19.458356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abb31f-750f-44ed-9cf6-bb39a2801a72 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:19.464028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:19.560882 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa26da5-7a4e-4e39-bbed-69fae3975da7 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:19.600542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:19.615658 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abb31f-750f-44ed-9cf6-bb39a2801a72 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:19.623077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cfdf8-374d-4b97-a3f1-cdefdbee08e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:19.624214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cfdf8-374d-4b97-a3f1-cdefdbee08e8 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:19.624765 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cfdf8-374d-4b97-a3f1-cdefdbee08e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:19.625279 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cfdf8-374d-4b97-a3f1-cdefdbee08e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:19.637882 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cfdf8-374d-4b97-a3f1-cdefdbee08e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:19.639333 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cfdf8-374d-4b97-a3f1-cdefdbee08e8 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:19.653506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-473cfdf8-374d-4b97-a3f1-cdefdbee08e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:19.653506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:19.653506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:19.653506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:19.653506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:19.653506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:19.653506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:19.653506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:19.653506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:19.653506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:19.653506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:19.653506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:19.653506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:19.653506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:19.653506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:19.662933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:19.662933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:19.662933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:19.662933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:19.662933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:19.662933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:19.662933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:19.662933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:19.662933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:19.662933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:19.662933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:19.662933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:19.662933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:19.662933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:19.662933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:19.662933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:19.662933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:19.664755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:19.664755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:19.664755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:19.664755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:19.664755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:19.664755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:19.664755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:19.664755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:19.664755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 271/529] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:19.664755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cfdf8-374d-4b97-a3f1-cdefdbee08e8 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:19.690242 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed9c44-90e0-4797-828a-86dab77cf075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:19.690968 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed9c44-90e0-4797-828a-86dab77cf075 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:19.691340 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed9c44-90e0-4797-828a-86dab77cf075 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:19.691635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed9c44-90e0-4797-828a-86dab77cf075 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:19.692640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed9c44-90e0-4797-828a-86dab77cf075 None admin] RBAC: Adding query filter params (role.id=6cff92a2d53f4855b38a9d9f11aa19bc, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=67bf3011e2bd47a3b5cf5e0b1173f065) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:19.692977 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed9c44-90e0-4797-828a-86dab77cf075 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:19.694812 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed9c44-90e0-4797-828a-86dab77cf075 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:19.701945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 259/530] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:58:19 2022] GET /identity/v3/role_assignments?user.id=67bf3011e2bd47a3b5cf5e0b1173f065&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=6cff92a2d53f4855b38a9d9f11aa19bc => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:19.702888 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed9c44-90e0-4797-828a-86dab77cf075 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:19.851466 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cfdf8-374d-4b97-a3f1-cdefdbee08e8 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:19.855753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:19.879200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed9c44-90e0-4797-828a-86dab77cf075 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:19.917871 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2aff9-d7df-430c-8050-191b8715e2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:19.918598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2aff9-d7df-430c-8050-191b8715e2fc None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:19.918972 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2aff9-d7df-430c-8050-191b8715e2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:19.919250 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2aff9-d7df-430c-8050-191b8715e2fc None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:19.920361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2aff9-d7df-430c-8050-191b8715e2fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:19.920793 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2aff9-d7df-430c-8050-191b8715e2fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:19.922614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2aff9-d7df-430c-8050-191b8715e2fc None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:19.942748 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a4018-1207-49d2-9b81-4d243e0b31f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:19.943449 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a4018-1207-49d2-9b81-4d243e0b31f5 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:19.943732 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a4018-1207-49d2-9b81-4d243e0b31f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:19.944049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a4018-1207-49d2-9b81-4d243e0b31f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:19.947797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 272/531] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:19.949542 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2aff9-d7df-430c-8050-191b8715e2fc None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:19.958057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a4018-1207-49d2-9b81-4d243e0b31f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:19.960229 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a4018-1207-49d2-9b81-4d243e0b31f5 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:19.978720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-577a4018-1207-49d2-9b81-4d243e0b31f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:19.978720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:19.978720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:19.978720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:19.978720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:19.978720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:19.978720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:19.978720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:19.978720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:19.978720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:19.978720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:19.978720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:19.978720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:19.978720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:19.978720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:19.979834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:19.979834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:19.979834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:19.979834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:19.979834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:19.979834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:19.979834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:19.979834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:19.979834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:19.979834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:19.979834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:19.979834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:19.979834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:19.979834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:19.979834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:19.979834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:19.979834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:19.980764 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:19.980764 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:19.980764 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:19.980764 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:19.980764 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 260/532] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:19.981184 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a4018-1207-49d2-9b81-4d243e0b31f5 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:20.006885 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:20.111007 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2aff9-d7df-430c-8050-191b8715e2fc None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:20.159272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a4018-1207-49d2-9b81-4d243e0b31f5 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:20.172232 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e5a83-4bb5-47d3-9062-a681e29d54c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:20.173676 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e5a83-4bb5-47d3-9062-a681e29d54c2 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:20.174291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e5a83-4bb5-47d3-9062-a681e29d54c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:20.174942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e5a83-4bb5-47d3-9062-a681e29d54c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:20.176589 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e5a83-4bb5-47d3-9062-a681e29d54c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:20.177383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e5a83-4bb5-47d3-9062-a681e29d54c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:20.180434 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e5a83-4bb5-47d3-9062-a681e29d54c2 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:20.202318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a24725-9fc0-4b83-bed2-f43e9580927d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:20.202968 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a24725-9fc0-4b83-bed2-f43e9580927d None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:20.203248 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a24725-9fc0-4b83-bed2-f43e9580927d None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:20.203523 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a24725-9fc0-4b83-bed2-f43e9580927d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:20.212148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a24725-9fc0-4b83-bed2-f43e9580927d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:20.212550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 273/533] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:20.213456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a24725-9fc0-4b83-bed2-f43e9580927d None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:20.214593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e5a83-4bb5-47d3-9062-a681e29d54c2 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:20.228171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-80a24725-9fc0-4b83-bed2-f43e9580927d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:20.228171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:20.228171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:20.228171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:20.228171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:20.228171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:20.228171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:20.228171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:20.228171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:20.228171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:20.228171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:20.228171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:20.228171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:20.228171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:20.228171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:20.230097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:20.230097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:20.230097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:20.230097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:20.230097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:20.230097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:20.230097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:20.230097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:20.230097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:20.230097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:20.230097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:20.230097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:20.230097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:20.230097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:20.230097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:20.230097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:20.230097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:20.231865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:20.231865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:20.231865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:20.231865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:20.231865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:20.231865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:20.231865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:20.231865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:20.231865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 261/534] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:20.231865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a24725-9fc0-4b83-bed2-f43e9580927d None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:20.261931 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:20.328955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:20.371566 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a24725-9fc0-4b83-bed2-f43e9580927d None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:20.412518 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e5a83-4bb5-47d3-9062-a681e29d54c2 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:20.421493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdb9b0-a8f4-4039-a2ee-d03f1dd3781e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:20.422174 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdb9b0-a8f4-4039-a2ee-d03f1dd3781e None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:20.422454 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdb9b0-a8f4-4039-a2ee-d03f1dd3781e None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:20.422727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdb9b0-a8f4-4039-a2ee-d03f1dd3781e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:20.423603 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cdb9b0-a8f4-4039-a2ee-d03f1dd3781e None admin] RBAC: Adding query filter params (role.id=6cff92a2d53f4855b38a9d9f11aa19bc, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=3d9be74621d14c94b95ae94947e5abbd) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:20.424071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cdb9b0-a8f4-4039-a2ee-d03f1dd3781e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:20.425817 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cdb9b0-a8f4-4039-a2ee-d03f1dd3781e None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:20.440530 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 262/535] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:58:20 2022] GET /identity/v3/role_assignments?user.id=3d9be74621d14c94b95ae94947e5abbd&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=6cff92a2d53f4855b38a9d9f11aa19bc => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:20.440530 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdb9b0-a8f4-4039-a2ee-d03f1dd3781e None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:20.514595 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa36c75-5f29-46cc-a35e-9a93d2cc5b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:20.515307 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa36c75-5f29-46cc-a35e-9a93d2cc5b38 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:20.515593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa36c75-5f29-46cc-a35e-9a93d2cc5b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:20.515900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa36c75-5f29-46cc-a35e-9a93d2cc5b38 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:20.516709 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa36c75-5f29-46cc-a35e-9a93d2cc5b38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:20.517018 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa36c75-5f29-46cc-a35e-9a93d2cc5b38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:20.520364 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa36c75-5f29-46cc-a35e-9a93d2cc5b38 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:20.558048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 274/536] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:20.561402 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa36c75-5f29-46cc-a35e-9a93d2cc5b38 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:20.731166 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdb9b0-a8f4-4039-a2ee-d03f1dd3781e None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:20.753760 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:20.775947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa36c75-5f29-46cc-a35e-9a93d2cc5b38 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:20.855303 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652c7dc-4802-4f2a-a67c-229e9c9aa708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:20.856588 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652c7dc-4802-4f2a-a67c-229e9c9aa708 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:20.857079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652c7dc-4802-4f2a-a67c-229e9c9aa708 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:20.857548 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652c7dc-4802-4f2a-a67c-229e9c9aa708 None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:20.859654 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb13da0-8a08-4920-8bd1-b6564faf631e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:20.860808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb13da0-8a08-4920-8bd1-b6564faf631e None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:20.861298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb13da0-8a08-4920-8bd1-b6564faf631e None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:20.861772 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb13da0-8a08-4920-8bd1-b6564faf631e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:20.863120 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652c7dc-4802-4f2a-a67c-229e9c9aa708 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:20.863734 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652c7dc-4802-4f2a-a67c-229e9c9aa708 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:20.866495 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652c7dc-4802-4f2a-a67c-229e9c9aa708 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:20.882397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb13da0-8a08-4920-8bd1-b6564faf631e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:20.885561 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb13da0-8a08-4920-8bd1-b6564faf631e None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:20.901074 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:20.932437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-deb13da0-8a08-4920-8bd1-b6564faf631e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:20.932437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:20.932437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:20.932437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:20.932437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:20.932437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:20.932437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:20.932437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:20.932437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:20.932437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:20.932437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:20.932437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:20.932437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:20.932437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:20.932437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:20.934205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:20.934205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:20.934205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:20.934205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:20.934205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:20.934205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:20.934205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:20.934205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:20.934205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:20.934205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:20.934205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:20.934205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:20.934205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:20.934205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:20.934205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:20.934205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:20.934205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:20.935743 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:20.935743 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:20.935743 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:20.935743 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:20.935743 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:20.935743 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:20.935743 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:20.935743 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:20.937485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 275/537] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:20.942388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 263/538] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 506 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:20.945982 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb13da0-8a08-4920-8bd1-b6564faf631e None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:20.950606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652c7dc-4802-4f2a-a67c-229e9c9aa708 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:21.202922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb13da0-8a08-4920-8bd1-b6564faf631e None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:21.270596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797a1fd-519e-4a22-ab25-7d8a137965a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:21.272151 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797a1fd-519e-4a22-ab25-7d8a137965a5 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:21.273365 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797a1fd-519e-4a22-ab25-7d8a137965a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:21.273999 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797a1fd-519e-4a22-ab25-7d8a137965a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:21.275948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f797a1fd-519e-4a22-ab25-7d8a137965a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:21.276637 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f797a1fd-519e-4a22-ab25-7d8a137965a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:21.280639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f797a1fd-519e-4a22-ab25-7d8a137965a5 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:21.288993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652c7dc-4802-4f2a-a67c-229e9c9aa708 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:21.308909 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 276/539] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:21.311772 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d2f08-4384-4b54-a24e-ce8c98d93659 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:21.313092 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d2f08-4384-4b54-a24e-ce8c98d93659 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:21.313722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d2f08-4384-4b54-a24e-ce8c98d93659 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:21.316911 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 277/540] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:21.319101 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d2f08-4384-4b54-a24e-ce8c98d93659 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:21.327150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:21.362553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8719d59-c1bd-4db7-9ae3-6ff3855684ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:21.363468 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8719d59-c1bd-4db7-9ae3-6ff3855684ea None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:21.363886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8719d59-c1bd-4db7-9ae3-6ff3855684ea None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:21.364328 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8719d59-c1bd-4db7-9ae3-6ff3855684ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:21.366659 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8719d59-c1bd-4db7-9ae3-6ff3855684ea None admin] RBAC: Adding query filter params (role.id=6cff92a2d53f4855b38a9d9f11aa19bc, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=4461a389da5848f5afd9854758707829) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:21.367226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8719d59-c1bd-4db7-9ae3-6ff3855684ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:21.369644 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8719d59-c1bd-4db7-9ae3-6ff3855684ea None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:21.379345 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 264/541] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:58:20 2022] GET /identity/v3/role_assignments?user.id=4461a389da5848f5afd9854758707829&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=6cff92a2d53f4855b38a9d9f11aa19bc => generated 259 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:21.383056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf675011-874f-4695-a538-13e4b447f878 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:21.383567 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf675011-874f-4695-a538-13e4b447f878 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:21.384003 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf675011-874f-4695-a538-13e4b447f878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:21.441699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-cf675011-874f-4695-a538-13e4b447f878 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:21.443537 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-cf675011-874f-4695-a538-13e4b447f878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:21.594000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:21.601385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d2f08-4384-4b54-a24e-ce8c98d93659 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:21.633493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 265/542] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:21.667064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40605f84-521d-4012-b357-23dc07ecb626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:21.667665 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40605f84-521d-4012-b357-23dc07ecb626 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:21.667980 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40605f84-521d-4012-b357-23dc07ecb626 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:21.668269 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40605f84-521d-4012-b357-23dc07ecb626 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:21.669080 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40605f84-521d-4012-b357-23dc07ecb626 None admin] RBAC: Adding query filter params (role.id=6cff92a2d53f4855b38a9d9f11aa19bc, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=4eba5f64833b482d81c53721c8442970) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:21.669463 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40605f84-521d-4012-b357-23dc07ecb626 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:21.670846 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40605f84-521d-4012-b357-23dc07ecb626 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:21.678027 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 278/543] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:58:21 2022] GET /identity/v3/role_assignments?user.id=4eba5f64833b482d81c53721c8442970&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=6cff92a2d53f4855b38a9d9f11aa19bc => generated 259 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:21.689801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf675011-874f-4695-a538-13e4b447f878 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:21.832508 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf675011-874f-4695-a538-13e4b447f878 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:21.878505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96194cb-26fc-4cb9-8795-40a9ae4f89f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:21.879184 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96194cb-26fc-4cb9-8795-40a9ae4f89f3 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:21.879466 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96194cb-26fc-4cb9-8795-40a9ae4f89f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:21.879740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96194cb-26fc-4cb9-8795-40a9ae4f89f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:21.893728 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96194cb-26fc-4cb9-8795-40a9ae4f89f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:21.899090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-b96194cb-26fc-4cb9-8795-40a9ae4f89f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:21.899090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:21.899090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:21.899090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:21.899090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:21.899090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:21.899090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:21.899090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:21.899090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:21.899090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:21.899090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:21.899090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:21.899090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:21.899090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:21.899090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:21.899090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:21.900087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:21.900087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:21.900087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:21.900087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:21.900087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:21.900087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:21.900087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:21.900087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:21.900087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:21.900087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:21.900087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:21.900087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:21.900087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:21.900087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:21.900087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:21.900087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:21.900941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:21.900941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:21.900941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:21.900941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:21.900941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:21.900941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:21.900941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:21.900941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:21.900941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:21.900941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:21.900941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:21.900941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:21.900941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 266/544] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:21.906748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40605f84-521d-4012-b357-23dc07ecb626 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:21.986207 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:22.031061 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40605f84-521d-4012-b357-23dc07ecb626 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:22.068307 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811da3b2-08ab-4764-9ca8-2a26eea679b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:22.068973 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811da3b2-08ab-4764-9ca8-2a26eea679b4 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:22.069580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811da3b2-08ab-4764-9ca8-2a26eea679b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:22.069922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811da3b2-08ab-4764-9ca8-2a26eea679b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:22.070644 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811da3b2-08ab-4764-9ca8-2a26eea679b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:22.071002 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811da3b2-08ab-4764-9ca8-2a26eea679b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:22.072394 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811da3b2-08ab-4764-9ca8-2a26eea679b4 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:22.081835 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 279/545] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 14:58:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:22.088242 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96194cb-26fc-4cb9-8795-40a9ae4f89f3 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:22.147758 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:22.218447 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96194cb-26fc-4cb9-8795-40a9ae4f89f3 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:22.264198 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a555363-acdb-449a-8f23-f3a4868e71c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:22.264917 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a555363-acdb-449a-8f23-f3a4868e71c1 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:22.265203 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a555363-acdb-449a-8f23-f3a4868e71c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:22.265488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a555363-acdb-449a-8f23-f3a4868e71c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:22.280013 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a555363-acdb-449a-8f23-f3a4868e71c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:22.281505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a555363-acdb-449a-8f23-f3a4868e71c1 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:22.303471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-4a555363-acdb-449a-8f23-f3a4868e71c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:22.303471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:22.303471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:22.303471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:22.303471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:22.303471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:22.303471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:22.303471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:22.303471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:22.303471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:22.303471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:22.303471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:22.303471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:22.303471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:22.303471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:22.304396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:22.304396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:22.304396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:22.304396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:22.304396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:22.304396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:22.304396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:22.304396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:22.304396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:22.304396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:22.304396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:22.304396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:22.304396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:22.304396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:22.304396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:22.304396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:22.304396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:22.305301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:22.305301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:22.305301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:22.305301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:22.305301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 267/546] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:22.310841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811da3b2-08ab-4764-9ca8-2a26eea679b4 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:22.374349 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ea2f7-f14f-4dc6-b0d9-d92fd081df94 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:22.374697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ea2f7-f14f-4dc6-b0d9-d92fd081df94 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:22.375022 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ea2f7-f14f-4dc6-b0d9-d92fd081df94 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:22.376494 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 268/547] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:58:22.384137 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e4bd3-7349-411d-95c5-38c7b4ff8aef None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:22.384137 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e4bd3-7349-411d-95c5-38c7b4ff8aef None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:22.384618 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e4bd3-7349-411d-95c5-38c7b4ff8aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:22.410629 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-e76e4bd3-7349-411d-95c5-38c7b4ff8aef None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:22.411880 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e76e4bd3-7349-411d-95c5-38c7b4ff8aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:22.510926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811da3b2-08ab-4764-9ca8-2a26eea679b4 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:22.540208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 269/548] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:58:22.559699 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c2d06-f162-4769-9520-7cbc1f188251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:22.560494 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c2d06-f162-4769-9520-7cbc1f188251 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:22.564662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:22.565143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c2d06-f162-4769-9520-7cbc1f188251 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:22.565755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c2d06-f162-4769-9520-7cbc1f188251 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:22.566740 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c2d06-f162-4769-9520-7cbc1f188251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:22.567313 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c2d06-f162-4769-9520-7cbc1f188251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:22.569727 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c2d06-f162-4769-9520-7cbc1f188251 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:22.579466 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e4bd3-7349-411d-95c5-38c7b4ff8aef None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:22.586835 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 280/549] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:22.592618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c2d06-f162-4769-9520-7cbc1f188251 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:22.614320 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:22.718945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e4bd3-7349-411d-95c5-38c7b4ff8aef None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:22.734320 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c2d06-f162-4769-9520-7cbc1f188251 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:22.781029 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c84169-09a8-4e16-91ea-6a9ed753fbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:22.781689 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c84169-09a8-4e16-91ea-6a9ed753fbf2 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:22.781971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c84169-09a8-4e16-91ea-6a9ed753fbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:22.782245 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c84169-09a8-4e16-91ea-6a9ed753fbf2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:22.791076 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc46495-8bdc-46e7-9387-48ba1a698adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:22.791731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc46495-8bdc-46e7-9387-48ba1a698adc None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:22.792018 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc46495-8bdc-46e7-9387-48ba1a698adc None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:22.792287 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc46495-8bdc-46e7-9387-48ba1a698adc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:22.795175 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c84169-09a8-4e16-91ea-6a9ed753fbf2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:22.796948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c84169-09a8-4e16-91ea-6a9ed753fbf2 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:22.811391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-a4c84169-09a8-4e16-91ea-6a9ed753fbf2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:58:22.811391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:22.811391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:22.811391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:22.811391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:22.811391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:22.811391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:22.811391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:22.811391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:22.811391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:22.811391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:22.811391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:22.811391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:22.811391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:22.811391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:22.811391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:22.812783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:22.812783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:22.812783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:22.812783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:22.812783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:22.812783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:22.812783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:22.812783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:22.812783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:22.812783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:22.812783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:22.812783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:22.812783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:22.812783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:22.812783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:22.812783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:22.812783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:22.812783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:22.814103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:22.814103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:22.814103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:22.814103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:22.814103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:22.814103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:22.814103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:22.814103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:22.814103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:58:22.814103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:22.814103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 281/550] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:58:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:58:22.819476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c84169-09a8-4e16-91ea-6a9ed753fbf2 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:22.821175 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc46495-8bdc-46e7-9387-48ba1a698adc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:22.826499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-6dc46495-8bdc-46e7-9387-48ba1a698adc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:22.826499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:22.826499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:22.826499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:22.826499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:22.826499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:22.826499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:22.826499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:22.826499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:22.826499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:22.826499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:22.826499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:22.826499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:22.826499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:22.826499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:22.826499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:22.827506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:22.827506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:22.827506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:22.827506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:22.827506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:22.827506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:22.827506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:22.827506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:22.827506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:22.827506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:22.827506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:22.827506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:22.827506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:22.827506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:22.827506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:22.827506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:22.828486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:22.828486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:22.828486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:22.828486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:22.828486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:22.828486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:22.828486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:22.828486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:22.828486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:22.828486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:22.828486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:22.828486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:22.831200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 270/551] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:22.836806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc46495-8bdc-46e7-9387-48ba1a698adc None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:22.910975 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:23.005008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:23.014449 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c84169-09a8-4e16-91ea-6a9ed753fbf2 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:23.018486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc46495-8bdc-46e7-9387-48ba1a698adc None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:23.059330 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e52d1c-a5f4-4572-9308-0114a0aaf30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:23.060337 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e52d1c-a5f4-4572-9308-0114a0aaf30a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:23.060873 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e52d1c-a5f4-4572-9308-0114a0aaf30a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:23.061805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e52d1c-a5f4-4572-9308-0114a0aaf30a None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:23.062721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e52d1c-a5f4-4572-9308-0114a0aaf30a None admin] RBAC: Adding query filter params (name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:23.063287 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e52d1c-a5f4-4572-9308-0114a0aaf30a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:23.064942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e52d1c-a5f4-4572-9308-0114a0aaf30a None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:23.083067 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 271/552] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 14:58:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:23.090430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e52d1c-a5f4-4572-9308-0114a0aaf30a None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:23.114455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08360e-e90d-4a7c-88b2-361212a3f37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:23.115614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08360e-e90d-4a7c-88b2-361212a3f37d None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:23.115917 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08360e-e90d-4a7c-88b2-361212a3f37d None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:23.116222 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08360e-e90d-4a7c-88b2-361212a3f37d None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:23.117135 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08360e-e90d-4a7c-88b2-361212a3f37d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:23.117445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08360e-e90d-4a7c-88b2-361212a3f37d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:23.118912 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08360e-e90d-4a7c-88b2-361212a3f37d None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:23.147246 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 282/553] 10.222.0.93 () {56 vars in 1135 bytes} [Mon Aug 1 14:58:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:58:23.148537 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5ecfb-2e1a-4e7c-83fb-7ad997dc6608 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:23.148828 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5ecfb-2e1a-4e7c-83fb-7ad997dc6608 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:23.149105 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5ecfb-2e1a-4e7c-83fb-7ad997dc6608 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:23.150847 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 283/554] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:23.152179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5ecfb-2e1a-4e7c-83fb-7ad997dc6608 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:23.233559 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e52d1c-a5f4-4572-9308-0114a0aaf30a None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:23.264986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5ecfb-2e1a-4e7c-83fb-7ad997dc6608 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:23.273142 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d6cea-b33f-40c9-80f2-8326cca6196b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:23.274135 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d6cea-b33f-40c9-80f2-8326cca6196b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:23.274840 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:23.275105 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d6cea-b33f-40c9-80f2-8326cca6196b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:23.275525 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d6cea-b33f-40c9-80f2-8326cca6196b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:23.290506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d6cea-b33f-40c9-80f2-8326cca6196b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:23.292281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d6cea-b33f-40c9-80f2-8326cca6196b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:23.301795 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a988b-65e4-445b-944d-e3db640b9c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:23.303377 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a988b-65e4-445b-944d-e3db640b9c8c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:23.303377 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a988b-65e4-445b-944d-e3db640b9c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:23.303565 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a988b-65e4-445b-944d-e3db640b9c8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:23.310705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-288d6cea-b33f-40c9-80f2-8326cca6196b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:23.310705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:23.310705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:23.310705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:23.310705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:23.310705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:23.310705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:23.310705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:23.310705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:23.310705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:23.310705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:23.310705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:23.310705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:23.310705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:23.310705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:23.311803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:23.311803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:23.311803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:23.311803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:23.311803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:23.311803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:23.311803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:23.311803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:23.311803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:23.311803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:23.311803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:23.311803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:23.311803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:23.311803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:23.311803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:23.311803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:23.311803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:23.312822 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:23.312822 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:23.312822 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:23.312822 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:23.314820 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 272/555] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:58:23.316387 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a67c33-4708-484b-85a8-89a28ba195a1 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:23.316872 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a67c33-4708-484b-85a8-89a28ba195a1 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:23.317397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a67c33-4708-484b-85a8-89a28ba195a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:23.331898 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:23.338887 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-96a67c33-4708-484b-85a8-89a28ba195a1 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:23.340282 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-96a67c33-4708-484b-85a8-89a28ba195a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:23.379787 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a988b-65e4-445b-944d-e3db640b9c8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:23.381949 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a988b-65e4-445b-944d-e3db640b9c8c None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:23.440486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 273/556] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:23.440486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a67c33-4708-484b-85a8-89a28ba195a1 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:23.522090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 284/557] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:58:23.523306 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a988b-65e4-445b-944d-e3db640b9c8c None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:23.534148 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:23.555871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a67c33-4708-484b-85a8-89a28ba195a1 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:23.596598 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ae9a7-4cc5-4fa2-8838-a28f4f72ed1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:23.597884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ae9a7-4cc5-4fa2-8838-a28f4f72ed1c None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:23.598658 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ae9a7-4cc5-4fa2-8838-a28f4f72ed1c None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:23.599951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ae9a7-4cc5-4fa2-8838-a28f4f72ed1c None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:23.601484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ae9a7-4cc5-4fa2-8838-a28f4f72ed1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:23.602331 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ae9a7-4cc5-4fa2-8838-a28f4f72ed1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:23.604724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ae9a7-4cc5-4fa2-8838-a28f4f72ed1c None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:23.607828 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:23.620451 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 274/558] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:23.622062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ae9a7-4cc5-4fa2-8838-a28f4f72ed1c None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:23.631922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a988b-65e4-445b-944d-e3db640b9c8c None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:23.668233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9324e6-2cb2-4158-843d-dbd23058ac2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:23.668912 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9324e6-2cb2-4158-843d-dbd23058ac2c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:23.669189 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9324e6-2cb2-4158-843d-dbd23058ac2c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:23.669470 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9324e6-2cb2-4158-843d-dbd23058ac2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:23.680543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9324e6-2cb2-4158-843d-dbd23058ac2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:23.686235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-2a9324e6-2cb2-4158-843d-dbd23058ac2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:23.686235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:23.686235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:23.686235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:23.686235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:23.686235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:23.686235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:23.686235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:23.686235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:23.686235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:23.686235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:23.686235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:23.686235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:23.686235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:23.686235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:23.686235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:23.687684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:23.687684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:23.687684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:23.687684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:23.687684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:23.687684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:23.687684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:23.687684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:23.687684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:23.687684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:23.687684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:23.687684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:23.687684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:23.687684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:23.687684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:23.687684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:23.688592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:23.688592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:23.688592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:23.688592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:23.688592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:23.688592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:23.688592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:23.688592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:23.688592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:23.688592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:23.688592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:23.688592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:23.692360 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 285/559] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:23.693340 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9324e6-2cb2-4158-843d-dbd23058ac2c None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:23.741840 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ae9a7-4cc5-4fa2-8838-a28f4f72ed1c None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:23.768829 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:23.787887 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764263f4-6790-4031-b4fc-002e08da44ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:23.788921 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764263f4-6790-4031-b4fc-002e08da44ad None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:23.789371 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764263f4-6790-4031-b4fc-002e08da44ad None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:23.789851 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764263f4-6790-4031-b4fc-002e08da44ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:23.803395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764263f4-6790-4031-b4fc-002e08da44ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:23.805274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764263f4-6790-4031-b4fc-002e08da44ad None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:23.810221 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9324e6-2cb2-4158-843d-dbd23058ac2c None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:23.822162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-764263f4-6790-4031-b4fc-002e08da44ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:23.822162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:23.822162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:23.822162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:23.822162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:23.822162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:23.822162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:23.822162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:23.822162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:23.822162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:23.822162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:23.822162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:23.822162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:23.822162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:23.822162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:23.823388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:23.823388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:23.823388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:23.823388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:23.823388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:23.823388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:23.823388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:23.823388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:23.823388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:23.823388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:23.823388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:23.823388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:23.823388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:23.823388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:23.823388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:23.823388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:23.823388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:23.824506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:23.824506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:23.824506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:23.824506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:23.826610 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 275/560] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:23.827632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764263f4-6790-4031-b4fc-002e08da44ad None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:23.848522 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba033f-b4e7-43b8-a707-5b69dcf113b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:23.849885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba033f-b4e7-43b8-a707-5b69dcf113b3 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:23.850158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba033f-b4e7-43b8-a707-5b69dcf113b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:23.850433 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba033f-b4e7-43b8-a707-5b69dcf113b3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:23.858270 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba033f-b4e7-43b8-a707-5b69dcf113b3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:23.859812 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba033f-b4e7-43b8-a707-5b69dcf113b3 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:23.873688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-73ba033f-b4e7-43b8-a707-5b69dcf113b3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 14:58:23.873688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:23.873688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:23.873688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:23.873688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:23.873688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:23.873688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:23.873688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:23.873688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:23.873688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:23.873688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:23.873688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:23.873688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:23.873688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:23.873688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:23.873688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:23.874952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:23.874952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:23.874952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:23.874952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:23.874952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:23.874952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:23.874952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:23.874952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:23.874952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:23.874952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:23.874952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:23.874952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:23.874952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:23.874952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:23.874952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:23.874952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:23.874952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:23.874952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:23.875979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:23.875979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:23.875979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:23.875979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:23.875979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:23.875979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:23.875979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:23.875979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:23.875979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 14:58:23.875979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:23.875979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 286/561] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:58:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:23.876705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba033f-b4e7-43b8-a707-5b69dcf113b3 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:23.891134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:23.960846 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764263f4-6790-4031-b4fc-002e08da44ad None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:24.013311 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93878a-2e5c-4398-a3d8-28b1afcc8e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:24.014254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93878a-2e5c-4398-a3d8-28b1afcc8e44 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:24.014716 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93878a-2e5c-4398-a3d8-28b1afcc8e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:24.015082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93878a-2e5c-4398-a3d8-28b1afcc8e44 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:24.016039 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93878a-2e5c-4398-a3d8-28b1afcc8e44 None admin] RBAC: Adding query filter params (name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:24.016588 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93878a-2e5c-4398-a3d8-28b1afcc8e44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:24.018761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93878a-2e5c-4398-a3d8-28b1afcc8e44 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:24.021761 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba033f-b4e7-43b8-a707-5b69dcf113b3 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:24.032568 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 276/562] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 14:58:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:24.034045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93878a-2e5c-4398-a3d8-28b1afcc8e44 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:24.059946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:24.071755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4012340-0067-45a0-a832-938302ed847c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:24.072590 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4012340-0067-45a0-a832-938302ed847c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:24.073032 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4012340-0067-45a0-a832-938302ed847c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:24.073572 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4012340-0067-45a0-a832-938302ed847c None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:24.074628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4012340-0067-45a0-a832-938302ed847c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:24.075445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4012340-0067-45a0-a832-938302ed847c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:24.076657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4012340-0067-45a0-a832-938302ed847c None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:24.090580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 287/563] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:24.091757 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4012340-0067-45a0-a832-938302ed847c None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:24.171025 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93878a-2e5c-4398-a3d8-28b1afcc8e44 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:24.183007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:24.214290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4580a2e-1338-4ea3-8209-aafdaba8491d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:24.214912 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4580a2e-1338-4ea3-8209-aafdaba8491d None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:24.215188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4580a2e-1338-4ea3-8209-aafdaba8491d None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:24.215467 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4580a2e-1338-4ea3-8209-aafdaba8491d None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:24.216291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4580a2e-1338-4ea3-8209-aafdaba8491d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:24.216642 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4580a2e-1338-4ea3-8209-aafdaba8491d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:24.218108 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4580a2e-1338-4ea3-8209-aafdaba8491d None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:24.219552 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4012340-0067-45a0-a832-938302ed847c None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:24.239520 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 277/564] 10.222.0.93 () {56 vars in 1137 bytes} [Mon Aug 1 14:58:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:24.246029 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9758f-edd3-4c29-a4cc-95b7ef9c5aa0 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:24.246399 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9758f-edd3-4c29-a4cc-95b7ef9c5aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:24.246636 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9758f-edd3-4c29-a4cc-95b7ef9c5aa0 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:24.248188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 278/565] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:24.252887 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe9758f-edd3-4c29-a4cc-95b7ef9c5aa0 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:24.265816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e5f41-f539-4ca0-9091-4191d9bf6903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:24.266605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e5f41-f539-4ca0-9091-4191d9bf6903 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:24.266916 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e5f41-f539-4ca0-9091-4191d9bf6903 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:24.267249 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e5f41-f539-4ca0-9091-4191d9bf6903 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:24.281505 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e5f41-f539-4ca0-9091-4191d9bf6903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:24.283177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e5f41-f539-4ca0-9091-4191d9bf6903 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:24.300596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-301e5f41-f539-4ca0-9091-4191d9bf6903 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:24.300596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:24.300596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:24.300596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:24.300596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:24.300596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:24.300596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:24.300596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:24.300596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:24.300596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:24.300596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:24.300596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:24.300596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:24.300596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:24.300596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:24.301573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:24.301573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:24.301573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:24.301573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:24.301573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:24.301573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:24.301573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:24.301573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:24.301573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:24.301573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:24.301573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:24.301573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:24.301573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:24.301573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:24.301573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:24.301573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:24.301573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:24.303036 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:24.303036 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:24.303036 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:24.303036 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:24.303036 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 288/566] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:24.303342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e5f41-f539-4ca0-9091-4191d9bf6903 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:24.354803 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:24.382096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe9758f-edd3-4c29-a4cc-95b7ef9c5aa0 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:24.420179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e5f41-f539-4ca0-9091-4191d9bf6903 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:24.422835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c2d9f-7bdc-40f5-9b71-b1c4026abc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:24.423552 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c2d9f-7bdc-40f5-9b71-b1c4026abc0c None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:24.423834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c2d9f-7bdc-40f5-9b71-b1c4026abc0c None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:24.424102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c2d9f-7bdc-40f5-9b71-b1c4026abc0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:24.432091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c2d9f-7bdc-40f5-9b71-b1c4026abc0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:24.433604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c2d9f-7bdc-40f5-9b71-b1c4026abc0c None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:24.445951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-6f7c2d9f-7bdc-40f5-9b71-b1c4026abc0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:24.445951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:24.445951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:24.445951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:24.445951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:24.445951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:24.445951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:24.445951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:24.445951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:24.445951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:24.445951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:24.445951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:24.445951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:24.445951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:24.445951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:24.447077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:24.447077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:24.447077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:24.447077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:24.447077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:24.447077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:24.447077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:24.447077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:24.447077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:24.447077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:24.447077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:24.447077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:24.447077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:24.447077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:24.447077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:24.447077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:24.447077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:24.448072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:24.448072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:24.448072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:24.448072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:24.448072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:24.448072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:24.448072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:24.448072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:24.448072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 279/567] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:24.448696 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da6426-df1c-4c4b-b19f-68061c6f78af None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:24.448696 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da6426-df1c-4c4b-b19f-68061c6f78af None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:24.448897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da6426-df1c-4c4b-b19f-68061c6f78af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:24.469339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5391454-619f-43cd-ada4-d228380f6bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:24.470239 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5391454-619f-43cd-ada4-d228380f6bfc None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:24.470589 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5391454-619f-43cd-ada4-d228380f6bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:24.471059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5391454-619f-43cd-ada4-d228380f6bfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:24.472075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-68da6426-df1c-4c4b-b19f-68061c6f78af None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:24.474002 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-68da6426-df1c-4c4b-b19f-68061c6f78af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:24.487899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5391454-619f-43cd-ada4-d228380f6bfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:24.490910 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5391454-619f-43cd-ada4-d228380f6bfc None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:24.510346 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:24.511422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-d5391454-619f-43cd-ada4-d228380f6bfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:24.511422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:24.511422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:24.511422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:24.511422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:24.511422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:24.511422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:24.511422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:24.511422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:24.511422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:24.511422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:24.511422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:24.511422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:24.511422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:24.511422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:24.512302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:24.512302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:24.512302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:24.512302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:24.512302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:24.512302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:24.512302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:24.512302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:24.512302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:24.512302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:24.512302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:24.512302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:24.512302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:24.512302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:24.512302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:24.512302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:24.512302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:24.513253 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:24.513253 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:24.513253 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:24.513253 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:24.513473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 289/568] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:24.514658 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5391454-619f-43cd-ada4-d228380f6bfc None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:24.558299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 280/569] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:24.561356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142bd15-8f33-4a96-8bd2-7466bfcc0b4c None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:24.561646 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142bd15-8f33-4a96-8bd2-7466bfcc0b4c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:24.562242 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142bd15-8f33-4a96-8bd2-7466bfcc0b4c None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:24.563711 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 281/570] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:24.564687 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142bd15-8f33-4a96-8bd2-7466bfcc0b4c None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:24.624452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:24.635764 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5391454-619f-43cd-ada4-d228380f6bfc None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:24.671365 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be8b50-86aa-4b8f-ada3-12d86446ee19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:24.672062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be8b50-86aa-4b8f-ada3-12d86446ee19 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:24.672349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be8b50-86aa-4b8f-ada3-12d86446ee19 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:24.672647 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be8b50-86aa-4b8f-ada3-12d86446ee19 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:24.673339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be8b50-86aa-4b8f-ada3-12d86446ee19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:24.673640 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be8b50-86aa-4b8f-ada3-12d86446ee19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:24.675129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be8b50-86aa-4b8f-ada3-12d86446ee19 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:24.684723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142bd15-8f33-4a96-8bd2-7466bfcc0b4c None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:24.688647 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 290/571] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:24.689513 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be8b50-86aa-4b8f-ada3-12d86446ee19 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:24.721179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f3334-222f-465f-9c00-3b5a8f32f91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:24.721985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f3334-222f-465f-9c00-3b5a8f32f91e None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:24.722261 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f3334-222f-465f-9c00-3b5a8f32f91e None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:24.722537 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f3334-222f-465f-9c00-3b5a8f32f91e None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:24.723288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f3334-222f-465f-9c00-3b5a8f32f91e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:24.723587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f3334-222f-465f-9c00-3b5a8f32f91e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:24.725586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f3334-222f-465f-9c00-3b5a8f32f91e None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:24.737361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 282/572] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:24.738577 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d46a3-50d7-40e7-a30f-a9db99bccb89 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:24.738861 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d46a3-50d7-40e7-a30f-a9db99bccb89 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:24.739139 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d46a3-50d7-40e7-a30f-a9db99bccb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:24.758463 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-9b6d46a3-50d7-40e7-a30f-a9db99bccb89 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:24.759665 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-9b6d46a3-50d7-40e7-a30f-a9db99bccb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:24.773866 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:24.796130 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be8b50-86aa-4b8f-ada3-12d86446ee19 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:24.831354 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 283/573] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:24.831565 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dac6c4-33ba-4268-926d-bbd3f6d23acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:24.832166 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dac6c4-33ba-4268-926d-bbd3f6d23acf None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:24.832440 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dac6c4-33ba-4268-926d-bbd3f6d23acf None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:24.832700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d46a3-50d7-40e7-a30f-a9db99bccb89 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:24.832810 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dac6c4-33ba-4268-926d-bbd3f6d23acf None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:24.833400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dac6c4-33ba-4268-926d-bbd3f6d23acf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:24.833699 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dac6c4-33ba-4268-926d-bbd3f6d23acf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:24.835062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dac6c4-33ba-4268-926d-bbd3f6d23acf None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:24.846945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 291/574] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:24.848009 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dac6c4-33ba-4268-926d-bbd3f6d23acf None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:24.866472 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:24.946844 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d46a3-50d7-40e7-a30f-a9db99bccb89 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:24.957710 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dac6c4-33ba-4268-926d-bbd3f6d23acf None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:24.984261 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f995102-8184-492e-9cf4-a817b0bae02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:24.984659 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f995102-8184-492e-9cf4-a817b0bae02d None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:24.984940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f995102-8184-492e-9cf4-a817b0bae02d None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:24.985207 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f995102-8184-492e-9cf4-a817b0bae02d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:24.995421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7c81a-1673-40aa-b429-f9287da4f881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:24.996353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7c81a-1673-40aa-b429-f9287da4f881 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:24.996992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7c81a-1673-40aa-b429-f9287da4f881 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:24.997162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f995102-8184-492e-9cf4-a817b0bae02d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:24.997352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7c81a-1673-40aa-b429-f9287da4f881 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:25.001913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-3f995102-8184-492e-9cf4-a817b0bae02d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:25.001913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:25.001913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:25.001913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:25.001913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:25.001913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:25.001913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:25.001913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:25.001913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:25.001913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:25.001913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:25.001913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:25.001913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:25.001913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:25.001913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:25.001913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:25.003745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:25.003745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:25.003745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:25.003745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:25.003745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:25.003745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:25.003745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:25.003745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:25.003745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:25.003745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:25.003745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:25.003745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:25.003745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:25.003745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:25.003745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:25.003745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:25.005478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:25.005478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:25.005478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:25.005478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:25.005478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:25.005478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:25.005478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:25.005478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:25.005478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:25.005478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:25.005478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:25.005478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:25.005478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 284/575] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:25.005478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f995102-8184-492e-9cf4-a817b0bae02d None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:25.011049 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7c81a-1673-40aa-b429-f9287da4f881 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:25.012811 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7c81a-1673-40aa-b429-f9287da4f881 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:25.022196 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:25.027582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-18d7c81a-1673-40aa-b429-f9287da4f881 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:58:25.027582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:25.027582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:25.027582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:25.027582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:25.027582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:25.027582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:25.027582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:25.027582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:25.027582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:25.027582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:25.027582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:25.027582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:25.027582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:25.027582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:25.027582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:25.028714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:25.028714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:25.028714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:25.028714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:25.028714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:25.028714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:25.028714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:25.028714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:25.028714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:25.028714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:25.028714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:25.028714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:25.028714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:25.028714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:25.028714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:25.028714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:25.028714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:25.028714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:25.029715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:25.029715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:25.029715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:25.029715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:25.029715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:25.029715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:25.029715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:25.029715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:25.029715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:58:25.029715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:25.029715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 292/576] 10.222.0.93 () {56 vars in 1093 bytes} [Mon Aug 1 14:58:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:25.029715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7c81a-1673-40aa-b429-f9287da4f881 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:25.109318 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:25.121550 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f995102-8184-492e-9cf4-a817b0bae02d None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:25.143097 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7c81a-1673-40aa-b429-f9287da4f881 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:25.158647 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0282fd-8abf-4bc9-8eaa-e7555504a186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:25.159436 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0282fd-8abf-4bc9-8eaa-e7555504a186 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:25.159711 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0282fd-8abf-4bc9-8eaa-e7555504a186 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:25.159983 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0282fd-8abf-4bc9-8eaa-e7555504a186 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:25.160816 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0282fd-8abf-4bc9-8eaa-e7555504a186 None admin] RBAC: Adding query filter params (role.id=6cff92a2d53f4855b38a9d9f11aa19bc, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=4df37b02152c48509243b73ebbf49d8d) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:25.161127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0282fd-8abf-4bc9-8eaa-e7555504a186 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:25.163300 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0282fd-8abf-4bc9-8eaa-e7555504a186 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:25.169831 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 285/577] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:58:25 2022] GET /identity/v3/role_assignments?user.id=4df37b02152c48509243b73ebbf49d8d&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=6cff92a2d53f4855b38a9d9f11aa19bc => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:25.170659 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0282fd-8abf-4bc9-8eaa-e7555504a186 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:25.186282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf50a3-e9b1-492d-8370-291818ea273d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:25.187077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf50a3-e9b1-492d-8370-291818ea273d None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:25.187584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf50a3-e9b1-492d-8370-291818ea273d None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:25.187817 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf50a3-e9b1-492d-8370-291818ea273d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:25.196251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf50a3-e9b1-492d-8370-291818ea273d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:25.198052 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf50a3-e9b1-492d-8370-291818ea273d None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:25.211848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-5edf50a3-e9b1-492d-8370-291818ea273d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:25.211848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:25.211848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:25.211848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:25.211848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:25.211848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:25.211848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:25.211848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:25.211848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:25.211848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:25.211848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:25.211848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:25.211848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:25.211848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:25.211848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:25.213430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:25.213430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:25.213430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:25.213430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:25.213430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:25.213430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:25.213430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:25.213430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:25.213430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:25.213430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:25.213430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:25.213430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:25.213430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:25.213430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:25.213430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:25.213430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:25.213430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:25.214926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:25.214926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:25.214926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:25.214926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:25.214926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:25.214926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:25.214926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:25.214926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:25.218233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 293/578] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:25.219222 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf50a3-e9b1-492d-8370-291818ea273d None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:25.271579 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:25.318476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0282fd-8abf-4bc9-8eaa-e7555504a186 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:25.356981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf50a3-e9b1-492d-8370-291818ea273d None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:25.361805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f9eca-e8cc-42ae-ad64-83f765efc7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:25.362454 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f9eca-e8cc-42ae-ad64-83f765efc7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:25.362779 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f9eca-e8cc-42ae-ad64-83f765efc7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:25.363056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f9eca-e8cc-42ae-ad64-83f765efc7e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:25.375627 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f9eca-e8cc-42ae-ad64-83f765efc7e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:25.381801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-e93f9eca-e8cc-42ae-ad64-83f765efc7e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:25.381801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:25.381801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:25.381801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:25.381801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:25.381801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:25.381801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:25.381801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:25.381801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:25.381801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:25.381801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:25.381801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:25.381801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:25.381801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:25.381801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:25.381801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:25.383091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:25.383091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:25.383091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:25.383091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:25.383091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:25.383091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:25.383091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:25.383091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:25.383091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:25.383091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:25.383091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:25.383091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:25.383091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:25.383091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:25.383091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:25.383091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:25.384171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:25.384171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:25.384171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:25.384171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:25.384171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:25.384171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:25.384171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:25.384171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:25.384171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:25.384171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:25.384171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:25.384171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:25.384171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 286/579] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:25.384171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f9eca-e8cc-42ae-ad64-83f765efc7e4 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:25.402381 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:25.410719 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b70ea-7bff-4cb0-bde1-7d48ae7317d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:25.411923 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b70ea-7bff-4cb0-bde1-7d48ae7317d4 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:25.412048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b70ea-7bff-4cb0-bde1-7d48ae7317d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:25.412897 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b70ea-7bff-4cb0-bde1-7d48ae7317d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:25.413530 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b70ea-7bff-4cb0-bde1-7d48ae7317d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:25.414035 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b70ea-7bff-4cb0-bde1-7d48ae7317d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:25.416609 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b70ea-7bff-4cb0-bde1-7d48ae7317d4 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:25.431852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 294/580] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 14:58:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:25.433969 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b70ea-7bff-4cb0-bde1-7d48ae7317d4 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:25.532765 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f9eca-e8cc-42ae-ad64-83f765efc7e4 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:25.558378 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b70ea-7bff-4cb0-bde1-7d48ae7317d4 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:25.569167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:25.576228 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e7bd0-398b-48a4-9a0f-e5f614efeb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:25.576886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e7bd0-398b-48a4-9a0f-e5f614efeb35 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:25.577290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e7bd0-398b-48a4-9a0f-e5f614efeb35 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:25.577454 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e7bd0-398b-48a4-9a0f-e5f614efeb35 None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:25.578259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e7bd0-398b-48a4-9a0f-e5f614efeb35 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:25.578572 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e7bd0-398b-48a4-9a0f-e5f614efeb35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:25.580052 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e7bd0-398b-48a4-9a0f-e5f614efeb35 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:25.597714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42814e79-3ef1-4621-b0cd-15044a053757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:25.598628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42814e79-3ef1-4621-b0cd-15044a053757 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:25.599110 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42814e79-3ef1-4621-b0cd-15044a053757 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:25.599565 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42814e79-3ef1-4621-b0cd-15044a053757 None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:25.600491 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42814e79-3ef1-4621-b0cd-15044a053757 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:25.600741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 287/581] 10.222.0.93 () {56 vars in 1133 bytes} [Mon Aug 1 14:58:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:25.601098 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42814e79-3ef1-4621-b0cd-15044a053757 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:25.601578 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e7bd0-398b-48a4-9a0f-e5f614efeb35 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:25.602985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42814e79-3ef1-4621-b0cd-15044a053757 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:25.618438 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 295/582] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:25.619763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42814e79-3ef1-4621-b0cd-15044a053757 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:25.699321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:25.741624 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e7bd0-398b-48a4-9a0f-e5f614efeb35 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:25.761116 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42814e79-3ef1-4621-b0cd-15044a053757 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:25.797657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0dfd87-36b7-48e2-908e-a3f167142b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:25.798379 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dfd87-36b7-48e2-908e-a3f167142b1f None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:25.798650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dfd87-36b7-48e2-908e-a3f167142b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:25.798932 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dfd87-36b7-48e2-908e-a3f167142b1f None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:25.799665 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0dfd87-36b7-48e2-908e-a3f167142b1f None admin] RBAC: Adding query filter params (name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:25.799988 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0dfd87-36b7-48e2-908e-a3f167142b1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:25.801489 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0dfd87-36b7-48e2-908e-a3f167142b1f None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:25.812921 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 288/583] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 14:58:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:25.813788 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0dfd87-36b7-48e2-908e-a3f167142b1f None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:25.821427 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188ccd0-0415-4902-b4e1-45993b3962fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:25.823905 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188ccd0-0415-4902-b4e1-45993b3962fc None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:25.824426 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188ccd0-0415-4902-b4e1-45993b3962fc None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:25.825314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188ccd0-0415-4902-b4e1-45993b3962fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:25.840175 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188ccd0-0415-4902-b4e1-45993b3962fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:25.842878 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188ccd0-0415-4902-b4e1-45993b3962fc None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:25.863184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-1188ccd0-0415-4902-b4e1-45993b3962fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:25.863184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:25.863184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:25.863184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:25.863184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:25.863184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:25.863184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:25.863184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:25.863184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:25.863184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:25.863184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:25.863184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:25.863184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:25.863184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:25.863184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:25.866805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:25.866805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:25.866805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:25.866805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:25.866805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:25.866805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:25.866805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:25.866805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:25.866805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:25.866805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:25.866805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:25.866805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:25.866805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:25.866805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:25.866805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:25.866805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:25.866805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:25.867775 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:25.867775 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:25.867775 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:25.867775 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:25.867775 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 296/584] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:25.867775 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188ccd0-0415-4902-b4e1-45993b3962fc None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:25.893915 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:25.960805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0dfd87-36b7-48e2-908e-a3f167142b1f None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:25.998127 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188ccd0-0415-4902-b4e1-45993b3962fc None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:26.002366 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca08999-8b53-45d9-85de-bf4ee194e2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:26.003138 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca08999-8b53-45d9-85de-bf4ee194e2bd None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:26.003515 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca08999-8b53-45d9-85de-bf4ee194e2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:26.003891 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca08999-8b53-45d9-85de-bf4ee194e2bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:26.007362 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:26.017838 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca08999-8b53-45d9-85de-bf4ee194e2bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:26.019676 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca08999-8b53-45d9-85de-bf4ee194e2bd None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:26.035280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-0ca08999-8b53-45d9-85de-bf4ee194e2bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:26.035280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:26.035280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:26.035280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:26.035280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:26.035280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:26.035280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:26.035280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:26.035280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:26.035280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:26.035280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:26.035280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:26.035280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:26.035280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:26.035280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:26.037944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:26.037944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:26.037944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:26.037944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:26.037944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:26.037944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:26.037944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:26.037944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:26.037944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:26.037944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:26.037944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:26.037944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:26.037944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:26.037944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:26.037944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:26.037944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:26.037944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:26.040179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:26.040179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:26.040179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:26.040179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:26.041651 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 289/585] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:26.042231 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca08999-8b53-45d9-85de-bf4ee194e2bd None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:26.047865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed2df0-5eb8-4a34-994f-c106737c3e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:26.048817 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed2df0-5eb8-4a34-994f-c106737c3e71 None admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:26.049279 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed2df0-5eb8-4a34-994f-c106737c3e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:26.049735 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed2df0-5eb8-4a34-994f-c106737c3e71 None admin] PATH_INFO: `/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/67bf3011e2bd47a3b5cf5e0b1173f065/roles/6cff92a2d53f4855b38a9d9f11aa19bc` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:26.072130 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed2df0-5eb8-4a34-994f-c106737c3e71 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac9f7c74597642b0acecc956429ac4d8, user_id=67bf3011e2bd47a3b5cf5e0b1173f065, role_id=6cff92a2d53f4855b38a9d9f11aa19bc)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:26.074078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed2df0-5eb8-4a34-994f-c106737c3e71 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:26.093775 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 297/586] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:58:25 2022] PUT /identity/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/67bf3011e2bd47a3b5cf5e0b1173f065/roles/6cff92a2d53f4855b38a9d9f11aa19bc => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:58:26.094650 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed2df0-5eb8-4a34-994f-c106737c3e71 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:26.166576 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca08999-8b53-45d9-85de-bf4ee194e2bd None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:26.171369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:26.210021 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7dc920-659e-4c0c-8378-4ede6d50f152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:26.210989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dc920-659e-4c0c-8378-4ede6d50f152 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:26.211455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dc920-659e-4c0c-8378-4ede6d50f152 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:26.211913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dc920-659e-4c0c-8378-4ede6d50f152 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:26.224850 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7dc920-659e-4c0c-8378-4ede6d50f152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:26.226657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7dc920-659e-4c0c-8378-4ede6d50f152 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:26.243907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-aa7dc920-659e-4c0c-8378-4ede6d50f152 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:26.243907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:26.243907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:26.243907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:26.243907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:26.243907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:26.243907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:26.243907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:26.243907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:26.243907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:26.243907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:26.243907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:26.243907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:26.243907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:26.243907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:26.244976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:26.244976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:26.244976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:26.244976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:26.244976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:26.244976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:26.244976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:26.244976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:26.244976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:26.244976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:26.244976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:26.244976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:26.244976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:26.244976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:26.244976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:26.244976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:26.244976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:26.246338 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:26.246338 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:26.246338 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:26.246338 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:26.247932 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 290/587] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:26.249051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7dc920-659e-4c0c-8378-4ede6d50f152 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:26.276814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed2df0-5eb8-4a34-994f-c106737c3e71 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:26.321931 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f443c81-7662-4a05-be58-4bb3af3046bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:26.322666 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f443c81-7662-4a05-be58-4bb3af3046bc None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:26.323115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f443c81-7662-4a05-be58-4bb3af3046bc None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:26.323500 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f443c81-7662-4a05-be58-4bb3af3046bc None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:26.324497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f443c81-7662-4a05-be58-4bb3af3046bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:26.324853 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f443c81-7662-4a05-be58-4bb3af3046bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:26.326426 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f443c81-7662-4a05-be58-4bb3af3046bc None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:26.341036 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 298/588] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:26.342099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f443c81-7662-4a05-be58-4bb3af3046bc None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:26.365712 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:26.375807 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7dc920-659e-4c0c-8378-4ede6d50f152 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:26.430386 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d6f1c-cc23-4ad7-94f4-4506d8464177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:26.431199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d6f1c-cc23-4ad7-94f4-4506d8464177 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:26.431581 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d6f1c-cc23-4ad7-94f4-4506d8464177 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:26.431971 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d6f1c-cc23-4ad7-94f4-4506d8464177 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:26.433074 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d6f1c-cc23-4ad7-94f4-4506d8464177 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:26.433481 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d6f1c-cc23-4ad7-94f4-4506d8464177 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:26.435947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d6f1c-cc23-4ad7-94f4-4506d8464177 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:26.454081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 291/589] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:26.455518 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d6f1c-cc23-4ad7-94f4-4506d8464177 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:26.477809 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f443c81-7662-4a05-be58-4bb3af3046bc None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:26.479213 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:26.530904 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c521e6-42b3-43eb-9519-378e8f036b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:26.531842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c521e6-42b3-43eb-9519-378e8f036b7d None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:26.532269 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c521e6-42b3-43eb-9519-378e8f036b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:26.532786 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c521e6-42b3-43eb-9519-378e8f036b7d None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:26.533652 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c521e6-42b3-43eb-9519-378e8f036b7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:26.534201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c521e6-42b3-43eb-9519-378e8f036b7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:26.535817 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c521e6-42b3-43eb-9519-378e8f036b7d None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:26.554831 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 299/590] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:26.555997 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c521e6-42b3-43eb-9519-378e8f036b7d None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:26.595649 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d6f1c-cc23-4ad7-94f4-4506d8464177 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:26.643018 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:26.644196 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8e6d4-5fa6-4f2b-bb33-a28302d10f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:26.645611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8e6d4-5fa6-4f2b-bb33-a28302d10f2f None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:26.646535 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8e6d4-5fa6-4f2b-bb33-a28302d10f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:26.647031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8e6d4-5fa6-4f2b-bb33-a28302d10f2f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:26.658276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8e6d4-5fa6-4f2b-bb33-a28302d10f2f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:26.660124 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8e6d4-5fa6-4f2b-bb33-a28302d10f2f None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:26.671072 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c521e6-42b3-43eb-9519-378e8f036b7d None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:26.677884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-42f8e6d4-5fa6-4f2b-bb33-a28302d10f2f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 14:58:26.677884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:26.677884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:26.677884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:26.677884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:26.677884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:26.677884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:26.677884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:26.677884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:26.677884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:26.677884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:26.677884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:26.677884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:26.677884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:26.677884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:26.677884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:26.679045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:26.679045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:26.679045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:26.679045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:26.679045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:26.679045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:26.679045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:26.679045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:26.679045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:26.679045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:26.679045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:26.679045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:26.679045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:26.679045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:26.679045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:26.679045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:26.679045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:26.679045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:26.680179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:26.680179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:26.680179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:26.680179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:26.680179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:26.680179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:26.680179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:26.680179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:26.680179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 14:58:26.680179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:26.684541 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 292/591] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:58:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:26.685637 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8e6d4-5fa6-4f2b-bb33-a28302d10f2f None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:26.717273 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b3ecf-b58f-4389-beb1-ae8a7cdce564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:26.718212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b3ecf-b58f-4389-beb1-ae8a7cdce564 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:26.718631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b3ecf-b58f-4389-beb1-ae8a7cdce564 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:26.719096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b3ecf-b58f-4389-beb1-ae8a7cdce564 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:26.728652 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b3ecf-b58f-4389-beb1-ae8a7cdce564 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:26.730327 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b3ecf-b58f-4389-beb1-ae8a7cdce564 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:26.742771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-df6b3ecf-b58f-4389-beb1-ae8a7cdce564 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:26.742771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:26.742771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:26.742771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:26.742771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:26.742771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:26.742771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:26.742771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:26.742771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:26.742771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:26.742771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:26.742771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:26.742771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:26.742771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:26.742771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:26.743888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:26.743888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:26.743888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:26.743888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:26.743888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:26.743888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:26.743888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:26.743888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:26.743888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:26.743888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:26.743888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:26.743888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:26.743888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:26.743888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:26.743888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:26.743888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:26.743888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:26.744881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:26.744881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:26.744881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:26.744881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:26.744881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:26.744881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:26.744881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:26.744881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:26.747292 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 300/592] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:26.748439 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b3ecf-b58f-4389-beb1-ae8a7cdce564 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:26.794880 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:26.828255 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8e6d4-5fa6-4f2b-bb33-a28302d10f2f None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:26.866683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300f51f-d54f-4dc9-947d-2b59754ea174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:26.867573 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300f51f-d54f-4dc9-947d-2b59754ea174 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:26.868000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300f51f-d54f-4dc9-947d-2b59754ea174 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:26.868557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300f51f-d54f-4dc9-947d-2b59754ea174 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:26.878446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300f51f-d54f-4dc9-947d-2b59754ea174 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:26.880947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300f51f-d54f-4dc9-947d-2b59754ea174 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:26.882116 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b3ecf-b58f-4389-beb1-ae8a7cdce564 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:26.895397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-7300f51f-d54f-4dc9-947d-2b59754ea174 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 14:58:26.895397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:26.895397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:26.895397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:26.895397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:26.895397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:26.895397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:26.895397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:26.895397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:26.895397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:26.895397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:26.895397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:26.895397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:26.895397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:26.895397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:26.895397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:26.897186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:26.897186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:26.897186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:26.897186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:26.897186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:26.897186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:26.897186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:26.897186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:26.897186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:26.897186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:26.897186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:26.897186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:26.897186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:26.897186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:26.897186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:26.897186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:26.897186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:26.897186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:26.898312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:26.898312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:26.898312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:26.898312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:26.898312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:26.898312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:26.898312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:26.898312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:26.898312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 14:58:26.898312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:26.905118 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 293/593] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:26.906464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300f51f-d54f-4dc9-947d-2b59754ea174 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:26.930857 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c5a7d-7b36-4a28-807a-f1999801224d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:26.931637 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c5a7d-7b36-4a28-807a-f1999801224d None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:26.932025 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c5a7d-7b36-4a28-807a-f1999801224d None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:26.933072 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c5a7d-7b36-4a28-807a-f1999801224d None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:26.933998 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c5a7d-7b36-4a28-807a-f1999801224d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:26.934694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c5a7d-7b36-4a28-807a-f1999801224d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:26.936573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c5a7d-7b36-4a28-807a-f1999801224d None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:26.942000 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:26.968666 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 301/594] 10.222.0.93 () {56 vars in 1137 bytes} [Mon Aug 1 14:58:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:26.968666 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c5a7d-7b36-4a28-807a-f1999801224d None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:27.050030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300f51f-d54f-4dc9-947d-2b59754ea174 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:27.077851 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c5a7d-7b36-4a28-807a-f1999801224d None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:27.100283 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e79d12-cb38-4ba5-a286-749232672093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:27.101604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e79d12-cb38-4ba5-a286-749232672093 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:27.102093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e79d12-cb38-4ba5-a286-749232672093 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:27.102826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e79d12-cb38-4ba5-a286-749232672093 None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:27.104512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e79d12-cb38-4ba5-a286-749232672093 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:27.105057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e79d12-cb38-4ba5-a286-749232672093 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:27.107622 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e79d12-cb38-4ba5-a286-749232672093 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:27.111631 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:27.115995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624748d-6db8-471e-8c81-e27b2abe7dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:27.116660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624748d-6db8-471e-8c81-e27b2abe7dfb None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:27.116936 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624748d-6db8-471e-8c81-e27b2abe7dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:27.117205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624748d-6db8-471e-8c81-e27b2abe7dfb None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:27.117992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624748d-6db8-471e-8c81-e27b2abe7dfb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:27.118294 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624748d-6db8-471e-8c81-e27b2abe7dfb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:27.120097 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624748d-6db8-471e-8c81-e27b2abe7dfb None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:27.126957 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 294/595] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:27.128375 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e79d12-cb38-4ba5-a286-749232672093 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:27.139700 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 302/596] 10.222.0.93 () {56 vars in 1139 bytes} [Mon Aug 1 14:58:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:27.145001 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624748d-6db8-471e-8c81-e27b2abe7dfb None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:27.216496 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:27.251415 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e79d12-cb38-4ba5-a286-749232672093 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:27.282329 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624748d-6db8-471e-8c81-e27b2abe7dfb None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:27.297489 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2024fd9-af56-4ce3-9457-3409f6a6b299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:27.298496 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2024fd9-af56-4ce3-9457-3409f6a6b299 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:27.299557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2024fd9-af56-4ce3-9457-3409f6a6b299 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:27.299939 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2024fd9-af56-4ce3-9457-3409f6a6b299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:27.311929 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2024fd9-af56-4ce3-9457-3409f6a6b299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:27.313324 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2024fd9-af56-4ce3-9457-3409f6a6b299 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:27.324974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ecb373-8fc6-4107-862b-ecc944550edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:27.325926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecb373-8fc6-4107-862b-ecc944550edb None admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:27.326376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecb373-8fc6-4107-862b-ecc944550edb None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:27.326800 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecb373-8fc6-4107-862b-ecc944550edb None admin] PATH_INFO: `/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/3d9be74621d14c94b95ae94947e5abbd/roles/6cff92a2d53f4855b38a9d9f11aa19bc` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:27.330533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-a2024fd9-af56-4ce3-9457-3409f6a6b299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:27.330533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:27.330533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:27.330533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:27.330533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:27.330533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:27.330533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:27.330533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:27.330533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:27.330533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:27.330533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:27.330533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:27.330533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:27.330533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:27.330533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:27.331441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:27.331441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:27.331441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:27.331441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:27.331441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:27.331441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:27.331441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:27.331441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:27.331441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:27.331441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:27.331441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:27.331441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:27.331441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:27.331441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:27.331441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:27.331441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:27.331441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:27.332374 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:27.332374 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:27.332374 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:27.332374 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:27.332374 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 295/597] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:27.332691 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2024fd9-af56-4ce3-9457-3409f6a6b299 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:27.345024 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ecb373-8fc6-4107-862b-ecc944550edb None admin] RBAC: Authorizing `identity:create_grant(project_id=ac9f7c74597642b0acecc956429ac4d8, user_id=3d9be74621d14c94b95ae94947e5abbd, role_id=6cff92a2d53f4855b38a9d9f11aa19bc)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:27.346700 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ecb373-8fc6-4107-862b-ecc944550edb None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:27.361763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 303/598] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:58:27 2022] PUT /identity/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/3d9be74621d14c94b95ae94947e5abbd/roles/6cff92a2d53f4855b38a9d9f11aa19bc => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:58:27.363390 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011e7e3-5d7c-4c82-a464-1703b1878eb9 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:27.363788 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011e7e3-5d7c-4c82-a464-1703b1878eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:27.363978 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011e7e3-5d7c-4c82-a464-1703b1878eb9 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:27.366245 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 304/599] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:27.366870 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011e7e3-5d7c-4c82-a464-1703b1878eb9 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:27.424911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:27.500960 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011e7e3-5d7c-4c82-a464-1703b1878eb9 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:27.502822 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:27.511948 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2024fd9-af56-4ce3-9457-3409f6a6b299 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:27.554471 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b84515-3430-4f16-9208-07cc7bb04aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:27.555466 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b84515-3430-4f16-9208-07cc7bb04aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:27.555580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b84515-3430-4f16-9208-07cc7bb04aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:27.555850 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b84515-3430-4f16-9208-07cc7bb04aa2 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:27.556581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b84515-3430-4f16-9208-07cc7bb04aa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:27.556891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b84515-3430-4f16-9208-07cc7bb04aa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:27.558595 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b84515-3430-4f16-9208-07cc7bb04aa2 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:27.560666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeb741-c11f-4b6b-92ac-a4f387188ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:27.561826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeb741-c11f-4b6b-92ac-a4f387188ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:27.562123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeb741-c11f-4b6b-92ac-a4f387188ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:27.562346 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeb741-c11f-4b6b-92ac-a4f387188ce0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:27.572205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 305/600] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:27.573509 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aed099-7053-41ec-b74f-6220d49b03f8 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:27.573903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aed099-7053-41ec-b74f-6220d49b03f8 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:27.574185 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aed099-7053-41ec-b74f-6220d49b03f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:27.576014 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeeb741-c11f-4b6b-92ac-a4f387188ce0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:27.577595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeeb741-c11f-4b6b-92ac-a4f387188ce0 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:27.597176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-3eeeb741-c11f-4b6b-92ac-a4f387188ce0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:27.597176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:27.597176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:27.597176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:27.597176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:27.597176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:27.597176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:27.597176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:27.597176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:27.597176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:27.597176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:27.597176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:27.597176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:27.597176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:27.597176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:27.598149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:27.598149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:27.598149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:27.598149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:27.598149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:27.598149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:27.598149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:27.598149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:27.598149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:27.598149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:27.598149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:27.598149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:27.598149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:27.598149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:27.598149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:27.598149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:27.598149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:27.599501 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:27.599501 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:27.599501 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:27.599501 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:27.599501 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 296/601] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:27.599793 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeb741-c11f-4b6b-92ac-a4f387188ce0 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:27.600086 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-87aed099-7053-41ec-b74f-6220d49b03f8 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:27.601364 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-87aed099-7053-41ec-b74f-6220d49b03f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:27.697894 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 306/602] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:27.698776 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aed099-7053-41ec-b74f-6220d49b03f8 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:27.732045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:27.739487 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeb741-c11f-4b6b-92ac-a4f387188ce0 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:27.779829 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85656ac5-bbd3-4645-be72-e6153ec7151b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:27.780497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85656ac5-bbd3-4645-be72-e6153ec7151b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:27.780788 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85656ac5-bbd3-4645-be72-e6153ec7151b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:27.781066 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85656ac5-bbd3-4645-be72-e6153ec7151b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:27.789302 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85656ac5-bbd3-4645-be72-e6153ec7151b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:27.790777 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85656ac5-bbd3-4645-be72-e6153ec7151b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:27.803881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-85656ac5-bbd3-4645-be72-e6153ec7151b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:27.803881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:27.803881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:27.803881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:27.803881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:27.803881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:27.803881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:27.803881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:27.803881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:27.803881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:27.803881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:27.803881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:27.803881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:27.803881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:27.803881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:27.805999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:27.805999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:27.805999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:27.805999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:27.805999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:27.805999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:27.805999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:27.805999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:27.805999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:27.805999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:27.805999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:27.805999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:27.805999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:27.805999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:27.805999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:27.805999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:27.805999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:27.808243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:27.808243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:27.808243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:27.808243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:27.808243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:27.808243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:27.808243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:27.808243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:27.809306 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:27.816829 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 297/603] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:27.816829 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85656ac5-bbd3-4645-be72-e6153ec7151b None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:27.818331 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aed099-7053-41ec-b74f-6220d49b03f8 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:27.858277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682d2db-cd72-45a5-94d6-9e34353b4277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:27.858907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682d2db-cd72-45a5-94d6-9e34353b4277 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:27.859286 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682d2db-cd72-45a5-94d6-9e34353b4277 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:27.859568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682d2db-cd72-45a5-94d6-9e34353b4277 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:27.860265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682d2db-cd72-45a5-94d6-9e34353b4277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:27.860610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682d2db-cd72-45a5-94d6-9e34353b4277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:27.862109 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682d2db-cd72-45a5-94d6-9e34353b4277 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:27.876731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 307/604] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:27.877739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682d2db-cd72-45a5-94d6-9e34353b4277 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:27.924507 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85656ac5-bbd3-4645-be72-e6153ec7151b None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:27.976813 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb3550-f7cd-44c5-bfbf-341d4d4d7d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:27.977514 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb3550-f7cd-44c5-bfbf-341d4d4d7d63 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:27.977837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb3550-f7cd-44c5-bfbf-341d4d4d7d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:27.978142 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb3550-f7cd-44c5-bfbf-341d4d4d7d63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:27.994460 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682d2db-cd72-45a5-94d6-9e34353b4277 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:27.995273 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb3550-f7cd-44c5-bfbf-341d4d4d7d63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:28.002707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-0cdb3550-f7cd-44c5-bfbf-341d4d4d7d63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:28.002707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:28.002707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:28.002707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:28.002707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:28.002707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:28.002707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:28.002707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:28.002707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:28.002707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:28.002707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:28.002707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:28.002707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:28.002707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:28.002707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:28.002707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:28.004049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:28.004049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:28.004049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:28.004049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:28.004049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:28.004049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:28.004049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:28.004049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:28.004049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:28.004049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:28.004049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:28.004049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:28.004049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:28.004049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:28.004049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:28.004049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:28.005220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:28.005220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:28.005220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:28.005220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:28.005220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:28.005220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:28.005220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:28.005220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:28.005220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:28.005220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:28.005220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:28.005220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:28.005220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 298/605] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:28.006233 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb3550-f7cd-44c5-bfbf-341d4d4d7d63 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:28.036422 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:28.044527 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960dd4a-1140-41ed-bf42-2f85f2b261e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:28.045659 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960dd4a-1140-41ed-bf42-2f85f2b261e4 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:28.046122 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960dd4a-1140-41ed-bf42-2f85f2b261e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:28.046544 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960dd4a-1140-41ed-bf42-2f85f2b261e4 None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:28.047604 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960dd4a-1140-41ed-bf42-2f85f2b261e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:28.048089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960dd4a-1140-41ed-bf42-2f85f2b261e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:28.049807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960dd4a-1140-41ed-bf42-2f85f2b261e4 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:28.065204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 308/606] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:28.066452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960dd4a-1140-41ed-bf42-2f85f2b261e4 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:28.084845 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:28.150953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb3550-f7cd-44c5-bfbf-341d4d4d7d63 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:28.200597 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960dd4a-1140-41ed-bf42-2f85f2b261e4 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:28.226393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76919e25-f395-495f-a6ff-aec473d031e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:28.227487 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76919e25-f395-495f-a6ff-aec473d031e8 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:28.227981 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76919e25-f395-495f-a6ff-aec473d031e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:28.229060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76919e25-f395-495f-a6ff-aec473d031e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:28.238971 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76919e25-f395-495f-a6ff-aec473d031e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:28.241274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76919e25-f395-495f-a6ff-aec473d031e8 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:28.254268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a248eb-f544-474b-98dc-01cafd3f1db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:28.255018 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a248eb-f544-474b-98dc-01cafd3f1db5 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:28.255965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a248eb-f544-474b-98dc-01cafd3f1db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:28.256308 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a248eb-f544-474b-98dc-01cafd3f1db5 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:28.257139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a248eb-f544-474b-98dc-01cafd3f1db5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:28.257511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a248eb-f544-474b-98dc-01cafd3f1db5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:28.258402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-76919e25-f395-495f-a6ff-aec473d031e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:28.258402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:28.258402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:28.258402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:28.258402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:28.258402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:28.258402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:28.258402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:28.258402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:28.258402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:28.258402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:28.258402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:28.258402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:28.258402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:28.258402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:28.259288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:28.259288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:28.259288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:28.259288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:28.259288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:28.259288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:28.259288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:28.259288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:28.259288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:28.259288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:28.259288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:28.259288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:28.259288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:28.259288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:28.259288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:28.259288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:28.259288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:28.260195 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:28.260195 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:28.260195 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:28.260195 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:28.260195 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:28.260195 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:28.260195 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:28.260195 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:28.260664 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a248eb-f544-474b-98dc-01cafd3f1db5 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:28.263332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 299/607] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:28.264298 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76919e25-f395-495f-a6ff-aec473d031e8 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:28.270264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 309/608] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 14:58:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:28.271799 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a248eb-f544-474b-98dc-01cafd3f1db5 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:28.358706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:28.376356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:28.397973 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a248eb-f544-474b-98dc-01cafd3f1db5 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:28.401925 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76919e25-f395-495f-a6ff-aec473d031e8 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:28.437507 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903f49d-2cc6-43c9-845e-fd1da09bcece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:28.438132 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903f49d-2cc6-43c9-845e-fd1da09bcece None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:28.438410 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903f49d-2cc6-43c9-845e-fd1da09bcece None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:28.438683 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903f49d-2cc6-43c9-845e-fd1da09bcece None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:28.439431 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903f49d-2cc6-43c9-845e-fd1da09bcece None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:28.439734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903f49d-2cc6-43c9-845e-fd1da09bcece None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:28.440872 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c7cbc-0eaf-43e1-8bc8-c576af606385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:28.441223 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903f49d-2cc6-43c9-845e-fd1da09bcece None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:28.441543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c7cbc-0eaf-43e1-8bc8-c576af606385 None admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:28.441820 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c7cbc-0eaf-43e1-8bc8-c576af606385 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:28.442091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c7cbc-0eaf-43e1-8bc8-c576af606385 None admin] PATH_INFO: `/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/4461a389da5848f5afd9854758707829/roles/6cff92a2d53f4855b38a9d9f11aa19bc` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:28.454102 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 310/609] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:28.454778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903f49d-2cc6-43c9-845e-fd1da09bcece None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:28.468450 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c7cbc-0eaf-43e1-8bc8-c576af606385 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac9f7c74597642b0acecc956429ac4d8, user_id=4461a389da5848f5afd9854758707829, role_id=6cff92a2d53f4855b38a9d9f11aa19bc)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:28.468450 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c7cbc-0eaf-43e1-8bc8-c576af606385 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:28.481828 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 300/610] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:58:28 2022] PUT /identity/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/4461a389da5848f5afd9854758707829/roles/6cff92a2d53f4855b38a9d9f11aa19bc => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:58:28.483411 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c0900-1ef2-46a6-8e71-7959d4535f09 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:28.483717 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c0900-1ef2-46a6-8e71-7959d4535f09 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:28.483989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c0900-1ef2-46a6-8e71-7959d4535f09 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:28.485635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 301/611] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:28.486261 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c0900-1ef2-46a6-8e71-7959d4535f09 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:28.602675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903f49d-2cc6-43c9-845e-fd1da09bcece None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:28.649636 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c0900-1ef2-46a6-8e71-7959d4535f09 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:28.657906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa68e97-f180-469e-85ef-e64ee06c7854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:28.658567 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa68e97-f180-469e-85ef-e64ee06c7854 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:28.658855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa68e97-f180-469e-85ef-e64ee06c7854 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:28.659135 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa68e97-f180-469e-85ef-e64ee06c7854 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:28.661834 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:28.675161 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa68e97-f180-469e-85ef-e64ee06c7854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:28.678068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa68e97-f180-469e-85ef-e64ee06c7854 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:28.693659 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:28.697445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-2aa68e97-f180-469e-85ef-e64ee06c7854 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:28.697445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:28.697445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:28.697445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:28.697445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:28.697445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:28.697445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:28.697445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:28.697445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:28.697445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:28.697445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:28.697445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:28.697445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:28.697445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:28.697445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:28.698580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:28.698580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:28.698580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:28.698580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:28.698580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:28.698580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:28.698580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:28.698580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:28.698580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:28.698580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:28.698580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:28.698580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:28.698580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:28.698580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:28.698580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:28.698580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:28.698580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:28.699480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:28.699480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:28.699480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:28.699480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:28.699480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 311/612] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:28.699770 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd7705-6d34-4f1e-a3fd-40241c89450d None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:28.700364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb095b4a-f813-4f03-9f73-563ff0e44cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:28.700518 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd7705-6d34-4f1e-a3fd-40241c89450d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:28.700518 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd7705-6d34-4f1e-a3fd-40241c89450d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:28.700858 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb095b4a-f813-4f03-9f73-563ff0e44cec None admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:28.704309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb095b4a-f813-4f03-9f73-563ff0e44cec None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:28.704309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb095b4a-f813-4f03-9f73-563ff0e44cec None admin] PATH_INFO: `/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/4eba5f64833b482d81c53721c8442970/roles/6cff92a2d53f4855b38a9d9f11aa19bc` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:28.722629 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb095b4a-f813-4f03-9f73-563ff0e44cec None admin] RBAC: Authorizing `identity:create_grant(project_id=ac9f7c74597642b0acecc956429ac4d8, user_id=4eba5f64833b482d81c53721c8442970, role_id=6cff92a2d53f4855b38a9d9f11aa19bc)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:28.724755 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb095b4a-f813-4f03-9f73-563ff0e44cec None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:28.725063 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-eefd7705-6d34-4f1e-a3fd-40241c89450d None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:28.726509 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-eefd7705-6d34-4f1e-a3fd-40241c89450d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:28.755375 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 302/613] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:58:28 2022] PUT /identity/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/4eba5f64833b482d81c53721c8442970/roles/6cff92a2d53f4855b38a9d9f11aa19bc => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:58:28.757192 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb095b4a-f813-4f03-9f73-563ff0e44cec None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:28.819904 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 312/614] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:28.845971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd7705-6d34-4f1e-a3fd-40241c89450d None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:28.902412 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb095b4a-f813-4f03-9f73-563ff0e44cec None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:28.966439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb3e87-d3a0-46bb-887a-b55370dd3af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:28.967696 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb3e87-d3a0-46bb-887a-b55370dd3af2 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:28.968749 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb3e87-d3a0-46bb-887a-b55370dd3af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:28.970109 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb3e87-d3a0-46bb-887a-b55370dd3af2 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:28.971101 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb3e87-d3a0-46bb-887a-b55370dd3af2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:28.971856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb3e87-d3a0-46bb-887a-b55370dd3af2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:28.975346 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb3e87-d3a0-46bb-887a-b55370dd3af2 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:29.005946 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 303/615] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:29.012075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb3e87-d3a0-46bb-887a-b55370dd3af2 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:29.020241 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd7705-6d34-4f1e-a3fd-40241c89450d None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:29.047694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:29.076934 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be79d34-ca4b-468f-9a2e-f1df7ef0a43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:29.077915 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be79d34-ca4b-468f-9a2e-f1df7ef0a43f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:29.078201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be79d34-ca4b-468f-9a2e-f1df7ef0a43f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:29.078478 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be79d34-ca4b-468f-9a2e-f1df7ef0a43f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:29.080380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:29.097070 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be79d34-ca4b-468f-9a2e-f1df7ef0a43f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:29.108947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-0be79d34-ca4b-468f-9a2e-f1df7ef0a43f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:29.108947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:29.108947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:29.108947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:29.108947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:29.108947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:29.108947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:29.108947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:29.108947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:29.108947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:29.108947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:29.108947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:29.108947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:29.108947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:29.108947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:29.108947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:29.109981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:29.109981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:29.109981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:29.109981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:29.109981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:29.109981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:29.109981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:29.109981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:29.109981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:29.109981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:29.109981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:29.109981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:29.109981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:29.109981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:29.109981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:29.109981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:29.110849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:29.110849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:29.110849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:29.110849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:29.110849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:29.110849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:29.110849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:29.110849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:29.110849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:29.110849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:29.110849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:29.110849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:29.110849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 313/616] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:58:29.115248 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be79d34-ca4b-468f-9a2e-f1df7ef0a43f None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:29.165286 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb3e87-d3a0-46bb-887a-b55370dd3af2 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:29.232238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231a837-ccea-4f63-85c6-793ec2c62459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:29.233784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231a837-ccea-4f63-85c6-793ec2c62459 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:29.234280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231a837-ccea-4f63-85c6-793ec2c62459 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:29.234850 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231a837-ccea-4f63-85c6-793ec2c62459 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:29.245081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231a837-ccea-4f63-85c6-793ec2c62459 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:29.247388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231a837-ccea-4f63-85c6-793ec2c62459 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:29.262217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-2231a837-ccea-4f63-85c6-793ec2c62459 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:58:29.262217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:29.262217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:29.262217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:29.262217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:29.262217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:29.262217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:29.262217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:29.262217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:29.262217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:29.262217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:29.262217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:29.262217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:29.262217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:29.262217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:29.262217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:29.263243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:29.263243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:29.263243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:29.263243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:29.263243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:29.263243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:29.263243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:29.263243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:29.263243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:29.263243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:29.263243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:29.263243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:29.263243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:29.263243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:29.263243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:29.263243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:29.263243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:29.263243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:29.264216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:29.264216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:29.264216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:29.264216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:29.264216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:29.264216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:29.264216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:29.264216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:29.264216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:58:29.264216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:29.267165 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 304/617] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:58:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:58:29.273440 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231a837-ccea-4f63-85c6-793ec2c62459 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:29.290570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be79d34-ca4b-468f-9a2e-f1df7ef0a43f None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:29.407350 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231a837-ccea-4f63-85c6-793ec2c62459 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:29.421034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:29.437342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad09f36-78ec-49e9-ab55-3f575496d850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:29.438093 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad09f36-78ec-49e9-ab55-3f575496d850 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:29.438573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad09f36-78ec-49e9-ab55-3f575496d850 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:29.439010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad09f36-78ec-49e9-ab55-3f575496d850 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:29.439944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad09f36-78ec-49e9-ab55-3f575496d850 None admin] RBAC: Adding query filter params (name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:29.440167 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad09f36-78ec-49e9-ab55-3f575496d850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:29.441798 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad09f36-78ec-49e9-ab55-3f575496d850 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:29.457852 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b292ad0-0dc7-46ad-9b75-1fc55452d517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:29.460377 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b292ad0-0dc7-46ad-9b75-1fc55452d517 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:29.461207 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b292ad0-0dc7-46ad-9b75-1fc55452d517 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:29.461713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b292ad0-0dc7-46ad-9b75-1fc55452d517 None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:29.463146 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b292ad0-0dc7-46ad-9b75-1fc55452d517 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:29.464057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b292ad0-0dc7-46ad-9b75-1fc55452d517 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:29.467002 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b292ad0-0dc7-46ad-9b75-1fc55452d517 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:29.475868 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 314/618] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 14:58:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:58:29.487194 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad09f36-78ec-49e9-ab55-3f575496d850 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:29.499011 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 305/619] 10.222.0.93 () {56 vars in 1135 bytes} [Mon Aug 1 14:58:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:29.506519 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b292ad0-0dc7-46ad-9b75-1fc55452d517 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:29.635573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:29.641204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b292ad0-0dc7-46ad-9b75-1fc55452d517 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:29.683073 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371640bd-4c4a-410d-8ea0-ad138a853928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:29.684301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371640bd-4c4a-410d-8ea0-ad138a853928 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:29.684844 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371640bd-4c4a-410d-8ea0-ad138a853928 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:29.685414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371640bd-4c4a-410d-8ea0-ad138a853928 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:29.698912 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371640bd-4c4a-410d-8ea0-ad138a853928 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:29.700775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371640bd-4c4a-410d-8ea0-ad138a853928 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:29.719312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-371640bd-4c4a-410d-8ea0-ad138a853928 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:29.719312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:29.719312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:29.719312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:29.719312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:29.719312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:29.719312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:29.719312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:29.719312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:29.719312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:29.719312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:29.719312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:29.719312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:29.719312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:29.719312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:29.720318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:29.720318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:29.720318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:29.720318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:29.720318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:29.720318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:29.720318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:29.720318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:29.720318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:29.720318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:29.720318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:29.720318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:29.720318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:29.720318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:29.720318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:29.720318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:29.720318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:29.721265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:29.721265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:29.721265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:29.721265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:29.724075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 306/620] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:29.730371 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371640bd-4c4a-410d-8ea0-ad138a853928 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:29.758359 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:29.805258 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad09f36-78ec-49e9-ab55-3f575496d850 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:29.854420 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fce0c7-1c39-4643-a061-fc922cffdb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:29.855161 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fce0c7-1c39-4643-a061-fc922cffdb97 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:29.855628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fce0c7-1c39-4643-a061-fc922cffdb97 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:29.855943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fce0c7-1c39-4643-a061-fc922cffdb97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:29.873656 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fce0c7-1c39-4643-a061-fc922cffdb97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:29.876146 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fce0c7-1c39-4643-a061-fc922cffdb97 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:29.896069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-a3fce0c7-1c39-4643-a061-fc922cffdb97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:29.896069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:29.896069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:29.896069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:29.896069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:29.896069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:29.896069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:29.896069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:29.896069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:29.896069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:29.896069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:29.896069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:29.896069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:29.896069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:29.896069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:29.897994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:29.897994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:29.897994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:29.897994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:29.897994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:29.897994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:29.897994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:29.897994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:29.897994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:29.897994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:29.897994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:29.897994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:29.897994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:29.897994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:29.897994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:29.897994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:29.897994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:29.899476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:29.899476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:29.899476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:29.899476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:29.899824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371640bd-4c4a-410d-8ea0-ad138a853928 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:29.903347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 315/621] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:29.905015 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41a0b8-88cb-4f0d-8478-a591b094e37e None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:29.905483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41a0b8-88cb-4f0d-8478-a591b094e37e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:29.905933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41a0b8-88cb-4f0d-8478-a591b094e37e None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:29.912188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 316/622] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:29.914168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41a0b8-88cb-4f0d-8478-a591b094e37e None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:29.970609 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c442d1-8bfb-4c1b-86ed-6eff175c808f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:29.971785 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c442d1-8bfb-4c1b-86ed-6eff175c808f None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:29.972496 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c442d1-8bfb-4c1b-86ed-6eff175c808f None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:29.972952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c442d1-8bfb-4c1b-86ed-6eff175c808f None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:29.974904 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c442d1-8bfb-4c1b-86ed-6eff175c808f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:29.975538 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c442d1-8bfb-4c1b-86ed-6eff175c808f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:29.978667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c442d1-8bfb-4c1b-86ed-6eff175c808f None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:30.005593 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 307/623] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:30.008192 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def13ff-68af-4614-a610-c6d1ca4a7cc8 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:30.009435 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def13ff-68af-4614-a610-c6d1ca4a7cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:30.009435 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def13ff-68af-4614-a610-c6d1ca4a7cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:30.041396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:30.047943 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-9def13ff-68af-4614-a610-c6d1ca4a7cc8 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:30.050182 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-9def13ff-68af-4614-a610-c6d1ca4a7cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:30.056218 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41a0b8-88cb-4f0d-8478-a591b094e37e None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:30.106461 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73d192-8fc5-47b0-9f98-78840d1f0728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:30.107930 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73d192-8fc5-47b0-9f98-78840d1f0728 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:30.108505 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73d192-8fc5-47b0-9f98-78840d1f0728 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:30.109386 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73d192-8fc5-47b0-9f98-78840d1f0728 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:30.110385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73d192-8fc5-47b0-9f98-78840d1f0728 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:30.111073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73d192-8fc5-47b0-9f98-78840d1f0728 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:30.115655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73d192-8fc5-47b0-9f98-78840d1f0728 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:30.139331 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 317/624] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:30.140703 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73d192-8fc5-47b0-9f98-78840d1f0728 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:30.181547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:30.188181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 308/625] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:30.190056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def13ff-68af-4614-a610-c6d1ca4a7cc8 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:30.315731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73d192-8fc5-47b0-9f98-78840d1f0728 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:30.345315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def13ff-68af-4614-a610-c6d1ca4a7cc8 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:30.371341 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7da64a-705d-4211-bed1-197d10bf3080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:30.373087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7da64a-705d-4211-bed1-197d10bf3080 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:30.373268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7da64a-705d-4211-bed1-197d10bf3080 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:30.373641 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7da64a-705d-4211-bed1-197d10bf3080 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:30.387211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7da64a-705d-4211-bed1-197d10bf3080 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:30.388771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7da64a-705d-4211-bed1-197d10bf3080 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:30.404809 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe64b33-a561-47e8-8dbf-ddfb53941916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:30.405870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe64b33-a561-47e8-8dbf-ddfb53941916 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:30.406158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-df7da64a-705d-4211-bed1-197d10bf3080 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:30.406158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:30.406158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:30.406158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:30.406158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:30.406158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:30.406158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:30.406158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:30.406158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:30.406158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:30.406158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:30.406158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:30.406158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:30.406158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:30.406158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:30.407164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:30.407164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:30.407164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:30.407164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:30.407164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:30.407164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:30.407164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:30.407164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:30.407164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:30.407164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:30.407164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:30.407164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:30.407164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:30.407164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:30.407164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:30.407164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:30.407164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:30.408742 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:30.408742 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:30.408742 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:30.408742 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:30.408742 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:30.408742 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:30.408742 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:30.408742 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:30.408742 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 318/626] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:30.408742 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7da64a-705d-4211-bed1-197d10bf3080 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:30.409785 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe64b33-a561-47e8-8dbf-ddfb53941916 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:30.410356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe64b33-a561-47e8-8dbf-ddfb53941916 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:30.421865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe64b33-a561-47e8-8dbf-ddfb53941916 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:30.424249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe64b33-a561-47e8-8dbf-ddfb53941916 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:30.424389 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:30.444122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-fbe64b33-a561-47e8-8dbf-ddfb53941916 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 14:58:30.444122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:30.444122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:30.444122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:30.444122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:30.444122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:30.444122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:30.444122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:30.444122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:30.444122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:30.444122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:30.444122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:30.444122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:30.444122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:30.444122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:30.444122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:30.445238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:30.445238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:30.445238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:30.445238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:30.445238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:30.445238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:30.445238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:30.445238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:30.445238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:30.445238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:30.445238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:30.445238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:30.445238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:30.445238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:30.445238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:30.445238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:30.445238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:30.445238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:30.446208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:30.446208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:30.446208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:30.446208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:30.446208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:30.446208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:30.446208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:30.446208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:30.446208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 14:58:30.446208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:30.449439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 309/627] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:58:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:30.450916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe64b33-a561-47e8-8dbf-ddfb53941916 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:30.549956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7da64a-705d-4211-bed1-197d10bf3080 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:30.550796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:30.598539 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c85bdf-32ee-4773-9fa3-e5e0672d6a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:30.600632 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c85bdf-32ee-4773-9fa3-e5e0672d6a56 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:30.601124 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c85bdf-32ee-4773-9fa3-e5e0672d6a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:30.602099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c85bdf-32ee-4773-9fa3-e5e0672d6a56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:30.611711 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe64b33-a561-47e8-8dbf-ddfb53941916 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:30.617462 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c85bdf-32ee-4773-9fa3-e5e0672d6a56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:30.627802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-04c85bdf-32ee-4773-9fa3-e5e0672d6a56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:30.627802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:30.627802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:30.627802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:30.627802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:30.627802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:30.627802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:30.627802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:30.627802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:30.627802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:30.627802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:30.627802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:30.627802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:30.627802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:30.627802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:30.627802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:30.628862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:30.628862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:30.628862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:30.628862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:30.628862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:30.628862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:30.628862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:30.628862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:30.628862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:30.628862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:30.628862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:30.628862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:30.628862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:30.628862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:30.628862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:30.628862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:30.630706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:30.630706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:30.630706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:30.630706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:30.630706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:30.630706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:30.630706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:30.630706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:30.630706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:30.630706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:30.630706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:30.630706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:30.637559 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 319/628] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:30.639243 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c85bdf-32ee-4773-9fa3-e5e0672d6a56 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:30.673748 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a39b01-297f-44c8-9d4f-d0739eb8796f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:30.675029 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a39b01-297f-44c8-9d4f-d0739eb8796f None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:30.675512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a39b01-297f-44c8-9d4f-d0739eb8796f None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:30.676530 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a39b01-297f-44c8-9d4f-d0739eb8796f None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:30.677945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a39b01-297f-44c8-9d4f-d0739eb8796f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:30.678545 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a39b01-297f-44c8-9d4f-d0739eb8796f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:30.680502 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a39b01-297f-44c8-9d4f-d0739eb8796f None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:30.705062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 310/629] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:30.706640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a39b01-297f-44c8-9d4f-d0739eb8796f None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:30.775411 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:30.828125 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c85bdf-32ee-4773-9fa3-e5e0672d6a56 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:30.852206 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a39b01-297f-44c8-9d4f-d0739eb8796f None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:30.871877 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51de47-86d8-4392-b82d-e379af1596f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:30.873054 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51de47-86d8-4392-b82d-e379af1596f0 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:30.873608 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51de47-86d8-4392-b82d-e379af1596f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:30.873889 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51de47-86d8-4392-b82d-e379af1596f0 None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:30.874770 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51de47-86d8-4392-b82d-e379af1596f0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:30.875074 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51de47-86d8-4392-b82d-e379af1596f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:30.876631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51de47-86d8-4392-b82d-e379af1596f0 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:30.888343 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:30.907062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a1a69-4471-4a9b-855d-913bc7d17edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:30.907672 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a1a69-4471-4a9b-855d-913bc7d17edc None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:30.907946 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a1a69-4471-4a9b-855d-913bc7d17edc None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:30.908213 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a1a69-4471-4a9b-855d-913bc7d17edc None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:30.913863 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 320/630] 10.222.0.93 () {56 vars in 1137 bytes} [Mon Aug 1 14:58:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:30.914076 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a1a69-4471-4a9b-855d-913bc7d17edc None admin] RBAC: Adding query filter params (name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:30.914076 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a1a69-4471-4a9b-855d-913bc7d17edc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:30.914354 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51de47-86d8-4392-b82d-e379af1596f0 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:30.914481 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a1a69-4471-4a9b-855d-913bc7d17edc None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:30.924851 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 311/631] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 14:58:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:30.928568 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06741647-ad5c-429c-96d4-6db802fc03db None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:30.929166 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06741647-ad5c-429c-96d4-6db802fc03db None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:30.929665 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06741647-ad5c-429c-96d4-6db802fc03db None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:30.934012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 312/632] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:30.934012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06741647-ad5c-429c-96d4-6db802fc03db None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:31.068429 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51de47-86d8-4392-b82d-e379af1596f0 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:31.069269 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06741647-ad5c-429c-96d4-6db802fc03db None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:31.100597 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:31.112200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be48237-fbe4-4380-82cb-35de81ca6e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:31.112802 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be48237-fbe4-4380-82cb-35de81ca6e6f None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:31.113076 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be48237-fbe4-4380-82cb-35de81ca6e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:31.113342 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be48237-fbe4-4380-82cb-35de81ca6e6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:31.124798 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f490763-371c-4cde-9f17-049a30807d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:31.125732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f490763-371c-4cde-9f17-049a30807d36 None admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:31.126081 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f490763-371c-4cde-9f17-049a30807d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:31.126435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f490763-371c-4cde-9f17-049a30807d36 None admin] PATH_INFO: `/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/4df37b02152c48509243b73ebbf49d8d/roles/6cff92a2d53f4855b38a9d9f11aa19bc` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:31.146475 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f490763-371c-4cde-9f17-049a30807d36 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac9f7c74597642b0acecc956429ac4d8, user_id=4df37b02152c48509243b73ebbf49d8d, role_id=6cff92a2d53f4855b38a9d9f11aa19bc)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:31.148493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f490763-371c-4cde-9f17-049a30807d36 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:31.164673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 321/633] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:58:30 2022] PUT /identity/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/4df37b02152c48509243b73ebbf49d8d/roles/6cff92a2d53f4855b38a9d9f11aa19bc => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:58:31.165610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f490763-371c-4cde-9f17-049a30807d36 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:31.200131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:31.232178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be48237-fbe4-4380-82cb-35de81ca6e6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:31.234755 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be48237-fbe4-4380-82cb-35de81ca6e6f None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:31.301760 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f490763-371c-4cde-9f17-049a30807d36 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:31.347327 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab7946-d5d6-463c-8458-bab760bae9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:31.348002 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab7946-d5d6-463c-8458-bab760bae9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:31.348281 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab7946-d5d6-463c-8458-bab760bae9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:31.348603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab7946-d5d6-463c-8458-bab760bae9f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:31.360735 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab7946-d5d6-463c-8458-bab760bae9f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:31.362602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab7946-d5d6-463c-8458-bab760bae9f9 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:31.384701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-b6ab7946-d5d6-463c-8458-bab760bae9f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:31.384701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:31.384701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:31.384701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:31.384701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:31.384701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:31.384701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:31.384701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:31.384701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:31.384701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:31.384701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:31.384701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:31.384701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:31.384701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:31.384701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:31.386153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:31.386153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:31.386153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:31.386153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:31.386153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:31.386153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:31.386153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:31.386153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:31.386153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:31.386153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:31.386153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:31.386153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:31.386153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:31.386153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:31.386153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:31.386153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:31.386153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:31.387447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:31.387447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:31.387447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:31.387447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:31.387447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 322/634] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:31.387788 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 313/635] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:58:31.387885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd7133-f885-4c93-bdeb-87b9b8fca4bc None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:31.387885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd7133-f885-4c93-bdeb-87b9b8fca4bc None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:31.387885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd7133-f885-4c93-bdeb-87b9b8fca4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:31.388094 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69289c26-c354-4ef1-811c-ca0c0bd76562 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:31.388815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69289c26-c354-4ef1-811c-ca0c0bd76562 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:31.389283 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69289c26-c354-4ef1-811c-ca0c0bd76562 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:31.391864 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 314/636] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:31.397571 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69289c26-c354-4ef1-811c-ca0c0bd76562 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:31.409611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-aadd7133-f885-4c93-bdeb-87b9b8fca4bc None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:31.410835 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-aadd7133-f885-4c93-bdeb-87b9b8fca4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:31.438835 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:31.500034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:31.512701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 323/637] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:31.512701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd7133-f885-4c93-bdeb-87b9b8fca4bc None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:31.537820 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69289c26-c354-4ef1-811c-ca0c0bd76562 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:31.575283 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5c06c-e333-4611-95b4-3477c3372ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:31.575901 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5c06c-e333-4611-95b4-3477c3372ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:31.576175 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5c06c-e333-4611-95b4-3477c3372ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:31.576443 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5c06c-e333-4611-95b4-3477c3372ac6 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:31.577161 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5c06c-e333-4611-95b4-3477c3372ac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:31.577458 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5c06c-e333-4611-95b4-3477c3372ac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:31.578846 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5c06c-e333-4611-95b4-3477c3372ac6 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:31.592679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 315/638] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:31.593973 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b067d5-5004-403e-9803-8639350c445c None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:31.594333 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b067d5-5004-403e-9803-8639350c445c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:31.594619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b067d5-5004-403e-9803-8639350c445c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:31.616256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-90b067d5-5004-403e-9803-8639350c445c None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:31.617765 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-90b067d5-5004-403e-9803-8639350c445c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:31.642790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd7133-f885-4c93-bdeb-87b9b8fca4bc None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:31.688104 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59204ce5-1ef3-4fd6-89c8-2bbcd64ab2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:31.689132 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59204ce5-1ef3-4fd6-89c8-2bbcd64ab2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:31.689451 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59204ce5-1ef3-4fd6-89c8-2bbcd64ab2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:31.689997 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59204ce5-1ef3-4fd6-89c8-2bbcd64ab2f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:31.702081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 316/639] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:31.702946 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b067d5-5004-403e-9803-8639350c445c None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:31.717004 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59204ce5-1ef3-4fd6-89c8-2bbcd64ab2f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:31.718503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59204ce5-1ef3-4fd6-89c8-2bbcd64ab2f2 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:31.739096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-59204ce5-1ef3-4fd6-89c8-2bbcd64ab2f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:31.739096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:31.739096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:31.739096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:31.739096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:31.739096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:31.739096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:31.739096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:31.739096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:31.739096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:31.739096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:31.739096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:31.739096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:31.739096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:31.739096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:31.741060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:31.741060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:31.741060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:31.741060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:31.741060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:31.741060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:31.741060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:31.741060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:31.741060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:31.741060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:31.741060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:31.741060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:31.741060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:31.741060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:31.741060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:31.741060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:31.741060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:31.742050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:31.742050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:31.742050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:31.742050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:31.745934 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 324/640] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:31.746897 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59204ce5-1ef3-4fd6-89c8-2bbcd64ab2f2 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:31.764867 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:31.817211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:31.861445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b067d5-5004-403e-9803-8639350c445c None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:31.892838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59204ce5-1ef3-4fd6-89c8-2bbcd64ab2f2 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:31.907901 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd8023-9989-4bad-b1de-3830db5e34fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:31.908642 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd8023-9989-4bad-b1de-3830db5e34fa None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:31.908930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd8023-9989-4bad-b1de-3830db5e34fa None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:31.909221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd8023-9989-4bad-b1de-3830db5e34fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:31.922867 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd8023-9989-4bad-b1de-3830db5e34fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:31.928079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-44bd8023-9989-4bad-b1de-3830db5e34fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:31.928079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:31.928079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:31.928079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:31.928079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:31.928079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:31.928079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:31.928079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:31.928079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:31.928079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:31.928079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:31.928079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:31.928079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:31.928079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:31.928079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:31.928079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:31.929452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:31.929452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:31.929452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:31.929452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:31.929452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:31.929452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:31.929452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:31.929452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:31.929452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:31.929452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:31.929452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:31.929452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:31.929452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:31.929452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:31.929452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:31.929452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:31.930557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:31.930557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:31.930557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:31.930557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:31.930557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:31.930557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:31.930557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:31.930557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:31.930557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:31.930557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:31.930557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:31.930557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:31.930557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 317/641] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:31.930557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd8023-9989-4bad-b1de-3830db5e34fa None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:31.944323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705e5ac-fc8c-4b48-83bf-c9f24a8a7035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:31.945409 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705e5ac-fc8c-4b48-83bf-c9f24a8a7035 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:31.945961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705e5ac-fc8c-4b48-83bf-c9f24a8a7035 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:31.946433 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705e5ac-fc8c-4b48-83bf-c9f24a8a7035 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:31.955960 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705e5ac-fc8c-4b48-83bf-c9f24a8a7035 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:31.957889 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705e5ac-fc8c-4b48-83bf-c9f24a8a7035 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:31.976352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-0705e5ac-fc8c-4b48-83bf-c9f24a8a7035 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:58:31.976352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:31.976352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:31.976352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:31.976352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:31.976352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:31.976352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:31.976352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:31.976352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:31.976352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:31.976352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:31.976352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:31.976352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:31.976352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:31.976352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:31.976352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:31.977675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:31.977675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:31.977675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:31.977675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:31.977675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:31.977675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:31.977675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:31.977675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:31.977675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:31.977675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:31.977675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:31.977675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:31.977675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:31.977675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:31.977675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:31.977675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:31.977675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:31.977675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:31.978921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:31.978921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:31.978921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:31.978921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:31.978921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:31.978921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:31.978921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:31.978921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:31.978921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:58:31.978921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:31.981309 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 325/642] 10.222.0.93 () {56 vars in 1093 bytes} [Mon Aug 1 14:58:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:31.982480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705e5ac-fc8c-4b48-83bf-c9f24a8a7035 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:32.075110 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd8023-9989-4bad-b1de-3830db5e34fa None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:32.085165 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:32.118948 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:32.127230 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6038f-13b7-4d7d-905a-68f7d4873d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:32.128337 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6038f-13b7-4d7d-905a-68f7d4873d80 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:32.128895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6038f-13b7-4d7d-905a-68f7d4873d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:32.129394 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6038f-13b7-4d7d-905a-68f7d4873d80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:32.135613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705e5ac-fc8c-4b48-83bf-c9f24a8a7035 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:32.150901 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6038f-13b7-4d7d-905a-68f7d4873d80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:32.160839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-dba6038f-13b7-4d7d-905a-68f7d4873d80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:32.160839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:32.160839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:32.160839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:32.160839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:32.160839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:32.160839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:32.160839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:32.160839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:32.160839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:32.160839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:32.160839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:32.160839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:32.160839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:32.160839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:32.160839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:32.162726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:32.162726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:32.162726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:32.162726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:32.162726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:32.162726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:32.162726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:32.162726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:32.162726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:32.162726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:32.162726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:32.162726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:32.162726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:32.162726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:32.162726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:32.162726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:32.165800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:32.165800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:32.165800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:32.165800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:32.165800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:32.165800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:32.165800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:32.165800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:32.165800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:32.165800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:32.165800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:32.165800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:32.165800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 318/643] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:32.165800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6038f-13b7-4d7d-905a-68f7d4873d80 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:32.214282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156e895-a490-42b1-99e8-cc1fadcb4aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:32.215111 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156e895-a490-42b1-99e8-cc1fadcb4aeb None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:32.215521 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156e895-a490-42b1-99e8-cc1fadcb4aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:32.215846 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156e895-a490-42b1-99e8-cc1fadcb4aeb None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:32.216602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156e895-a490-42b1-99e8-cc1fadcb4aeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:32.216922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156e895-a490-42b1-99e8-cc1fadcb4aeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:32.218465 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156e895-a490-42b1-99e8-cc1fadcb4aeb None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:32.233262 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 326/644] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:32.234313 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156e895-a490-42b1-99e8-cc1fadcb4aeb None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:32.369670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6038f-13b7-4d7d-905a-68f7d4873d80 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:32.406721 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156e895-a490-42b1-99e8-cc1fadcb4aeb None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:32.422765 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19a736-1748-420b-bcf3-f339af3f996f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:32.423417 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19a736-1748-420b-bcf3-f339af3f996f None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:32.423801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19a736-1748-420b-bcf3-f339af3f996f None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:32.424038 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19a736-1748-420b-bcf3-f339af3f996f None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:32.424804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19a736-1748-420b-bcf3-f339af3f996f None admin] RBAC: Adding query filter params (name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:32.425050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19a736-1748-420b-bcf3-f339af3f996f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:32.426529 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19a736-1748-420b-bcf3-f339af3f996f None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:32.442313 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 319/645] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 14:58:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:32.442313 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19a736-1748-420b-bcf3-f339af3f996f None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:32.453910 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:32.461149 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb53e11-04b6-4e21-bb25-adbaa6e9d461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:32.462233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb53e11-04b6-4e21-bb25-adbaa6e9d461 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:32.462857 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb53e11-04b6-4e21-bb25-adbaa6e9d461 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:32.463151 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb53e11-04b6-4e21-bb25-adbaa6e9d461 None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:32.464180 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb53e11-04b6-4e21-bb25-adbaa6e9d461 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:32.464525 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb53e11-04b6-4e21-bb25-adbaa6e9d461 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:32.466319 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb53e11-04b6-4e21-bb25-adbaa6e9d461 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:32.490784 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 327/646] 10.222.0.93 () {56 vars in 1133 bytes} [Mon Aug 1 14:58:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:32.496333 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb53e11-04b6-4e21-bb25-adbaa6e9d461 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:32.535849 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:32.595997 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19a736-1748-420b-bcf3-f339af3f996f None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:32.633842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb53e11-04b6-4e21-bb25-adbaa6e9d461 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:32.642066 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed5b8e-d488-443e-ad6b-569279cd3fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:32.642756 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed5b8e-d488-443e-ad6b-569279cd3fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:32.643062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed5b8e-d488-443e-ad6b-569279cd3fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:32.643421 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed5b8e-d488-443e-ad6b-569279cd3fa6 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:32.644097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed5b8e-d488-443e-ad6b-569279cd3fa6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:32.644437 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed5b8e-d488-443e-ad6b-569279cd3fa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:32.646005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed5b8e-d488-443e-ad6b-569279cd3fa6 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:32.661733 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 320/647] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 14:58:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:32.661733 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed5b8e-d488-443e-ad6b-569279cd3fa6 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:32.688139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33647151-fec3-49e0-9acd-0aceb481ad96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:32.689050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33647151-fec3-49e0-9acd-0aceb481ad96 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:32.689451 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33647151-fec3-49e0-9acd-0aceb481ad96 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:32.689752 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33647151-fec3-49e0-9acd-0aceb481ad96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:32.699783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33647151-fec3-49e0-9acd-0aceb481ad96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:32.701354 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33647151-fec3-49e0-9acd-0aceb481ad96 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:32.716593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-33647151-fec3-49e0-9acd-0aceb481ad96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:32.716593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:32.716593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:32.716593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:32.716593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:32.716593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:32.716593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:32.716593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:32.716593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:32.716593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:32.716593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:32.716593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:32.716593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:32.716593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:32.716593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:32.718216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:32.718216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:32.718216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:32.718216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:32.718216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:32.718216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:32.718216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:32.718216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:32.718216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:32.718216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:32.718216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:32.718216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:32.718216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:32.718216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:32.718216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:32.718216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:32.718216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:32.719701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:32.719701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:32.719701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:32.719701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:32.719701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:32.719701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:32.719701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:32.719701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:32.719701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 328/648] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:32.719701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33647151-fec3-49e0-9acd-0aceb481ad96 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:32.789426 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:32.807533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed5b8e-d488-443e-ad6b-569279cd3fa6 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:32.828351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:32.855977 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4949f00-0bb1-4222-b51c-4f5090f006b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:32.856655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4949f00-0bb1-4222-b51c-4f5090f006b5 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:32.856934 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4949f00-0bb1-4222-b51c-4f5090f006b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:32.857203 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4949f00-0bb1-4222-b51c-4f5090f006b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:32.870927 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4949f00-0bb1-4222-b51c-4f5090f006b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:32.871157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4949f00-0bb1-4222-b51c-4f5090f006b5 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:32.874432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33647151-fec3-49e0-9acd-0aceb481ad96 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:32.889220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-e4949f00-0bb1-4222-b51c-4f5090f006b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:32.889220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:32.889220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:32.889220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:32.889220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:32.889220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:32.889220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:32.889220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:32.889220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:32.889220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:32.889220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:32.889220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:32.889220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:32.889220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:32.889220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:32.890391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:32.890391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:32.890391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:32.890391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:32.890391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:32.890391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:32.890391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:32.890391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:32.890391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:32.890391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:32.890391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:32.890391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:32.890391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:32.890391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:32.890391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:32.890391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:32.890391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:32.891524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:32.891524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:32.891524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:32.891524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:32.891524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 321/649] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:32.892021 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4949f00-0bb1-4222-b51c-4f5090f006b5 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:32.925691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d110c9-7138-430f-8921-4ccbec8643dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:32.926695 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d110c9-7138-430f-8921-4ccbec8643dc None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:32.927160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d110c9-7138-430f-8921-4ccbec8643dc None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:32.927610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d110c9-7138-430f-8921-4ccbec8643dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:33.008889 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d110c9-7138-430f-8921-4ccbec8643dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:33.010414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d110c9-7138-430f-8921-4ccbec8643dc None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:33.020081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4949f00-0bb1-4222-b51c-4f5090f006b5 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:33.068684 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0784e-2af5-4ca6-9604-7148b71d6b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:33.069671 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0784e-2af5-4ca6-9604-7148b71d6b17 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:33.070220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0784e-2af5-4ca6-9604-7148b71d6b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:33.070661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0784e-2af5-4ca6-9604-7148b71d6b17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:33.076838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:33.088536 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0784e-2af5-4ca6-9604-7148b71d6b17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:33.090354 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0784e-2af5-4ca6-9604-7148b71d6b17 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:33.107606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-74f0784e-2af5-4ca6-9604-7148b71d6b17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:33.107606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:33.107606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:33.107606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:33.107606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:33.107606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:33.107606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:33.107606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:33.107606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:33.107606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:33.107606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:33.107606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:33.107606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:33.107606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:33.107606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:33.109699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:33.109699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:33.109699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:33.109699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:33.109699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:33.109699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:33.109699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:33.109699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:33.109699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:33.109699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:33.109699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:33.109699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:33.109699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:33.109699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:33.109699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:33.109699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:33.109699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:33.111539 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:33.111539 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:33.111539 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:33.111539 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:33.115950 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 322/650] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:33.117111 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0784e-2af5-4ca6-9604-7148b71d6b17 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:33.146550 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:33.147108 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 329/651] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:58:33.148045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d110c9-7138-430f-8921-4ccbec8643dc None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:33.259846 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0784e-2af5-4ca6-9604-7148b71d6b17 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:33.284858 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d110c9-7138-430f-8921-4ccbec8643dc None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:33.300932 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4558e17-67d3-49e9-8dfb-a0e49c15cb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:33.301461 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4558e17-67d3-49e9-8dfb-a0e49c15cb4c None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:33.301716 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4558e17-67d3-49e9-8dfb-a0e49c15cb4c None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:33.302496 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4558e17-67d3-49e9-8dfb-a0e49c15cb4c None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:33.303376 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4558e17-67d3-49e9-8dfb-a0e49c15cb4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:33.303489 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4558e17-67d3-49e9-8dfb-a0e49c15cb4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:33.305070 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4558e17-67d3-49e9-8dfb-a0e49c15cb4c None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:33.320375 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 323/652] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:33.321252 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4558e17-67d3-49e9-8dfb-a0e49c15cb4c None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:33.337981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6554b8-71c8-471a-99eb-85f36902838f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:33.339103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6554b8-71c8-471a-99eb-85f36902838f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:33.339967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6554b8-71c8-471a-99eb-85f36902838f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:33.340597 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6554b8-71c8-471a-99eb-85f36902838f None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:33.341631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6554b8-71c8-471a-99eb-85f36902838f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:33.342314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6554b8-71c8-471a-99eb-85f36902838f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:33.344266 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6554b8-71c8-471a-99eb-85f36902838f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:33.366843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 330/653] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:33.366843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6554b8-71c8-471a-99eb-85f36902838f None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:33.381529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:33.419226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:33.443466 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4558e17-67d3-49e9-8dfb-a0e49c15cb4c None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:33.483560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ef3e2-3e92-4656-9156-b2cbb978c318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:33.484201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ef3e2-3e92-4656-9156-b2cbb978c318 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:33.484556 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ef3e2-3e92-4656-9156-b2cbb978c318 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:33.484811 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ef3e2-3e92-4656-9156-b2cbb978c318 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:33.485446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ef3e2-3e92-4656-9156-b2cbb978c318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:33.485812 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ef3e2-3e92-4656-9156-b2cbb978c318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:33.487151 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ef3e2-3e92-4656-9156-b2cbb978c318 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:33.493954 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6554b8-71c8-471a-99eb-85f36902838f None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:33.509075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 324/654] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:33.509075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ef3e2-3e92-4656-9156-b2cbb978c318 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:33.546257 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf718043-d9b6-4416-90b6-a3e1bed0bbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:33.546966 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf718043-d9b6-4416-90b6-a3e1bed0bbed None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:33.547468 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf718043-d9b6-4416-90b6-a3e1bed0bbed None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:33.547709 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf718043-d9b6-4416-90b6-a3e1bed0bbed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:33.562040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf718043-d9b6-4416-90b6-a3e1bed0bbed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:33.563933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf718043-d9b6-4416-90b6-a3e1bed0bbed None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:33.579356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-cf718043-d9b6-4416-90b6-a3e1bed0bbed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:33.579356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:33.579356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:33.579356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:33.579356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:33.579356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:33.579356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:33.579356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:33.579356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:33.579356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:33.579356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:33.579356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:33.579356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:33.579356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:33.579356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:33.580691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:33.580691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:33.580691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:33.580691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:33.580691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:33.580691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:33.580691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:33.580691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:33.580691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:33.580691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:33.580691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:33.580691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:33.580691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:33.580691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:33.580691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:33.580691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:33.580691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:33.581816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:33.581816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:33.581816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:33.581816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:33.584634 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 331/655] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:33.585741 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf718043-d9b6-4416-90b6-a3e1bed0bbed None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:33.620048 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ef3e2-3e92-4656-9156-b2cbb978c318 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:33.658855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c63b20-a834-4b6b-a3df-aa4c11f62e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:33.659522 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c63b20-a834-4b6b-a3df-aa4c11f62e20 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:33.659899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c63b20-a834-4b6b-a3df-aa4c11f62e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:33.660210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c63b20-a834-4b6b-a3df-aa4c11f62e20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:33.661164 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c63b20-a834-4b6b-a3df-aa4c11f62e20 None admin] RBAC: Adding query filter params (role.id=6cff92a2d53f4855b38a9d9f11aa19bc, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=67bf3011e2bd47a3b5cf5e0b1173f065) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:33.661525 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c63b20-a834-4b6b-a3df-aa4c11f62e20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:33.663162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c63b20-a834-4b6b-a3df-aa4c11f62e20 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:33.668661 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:33.680557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 325/656] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:58:33 2022] GET /identity/v3/role_assignments?user.id=67bf3011e2bd47a3b5cf5e0b1173f065&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=6cff92a2d53f4855b38a9d9f11aa19bc => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:33.680557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb883ba-6846-40e5-8398-2f4912ce8542 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:33.680557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb883ba-6846-40e5-8398-2f4912ce8542 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:33.680557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb883ba-6846-40e5-8398-2f4912ce8542 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:33.680557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 326/657] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:33.680557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb883ba-6846-40e5-8398-2f4912ce8542 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:33.681206 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:33.742123 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf718043-d9b6-4416-90b6-a3e1bed0bbed None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:33.791733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7034b90-effc-43f2-acdb-1ebfb5a994c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:33.792368 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7034b90-effc-43f2-acdb-1ebfb5a994c7 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:33.792704 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7034b90-effc-43f2-acdb-1ebfb5a994c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:33.792964 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7034b90-effc-43f2-acdb-1ebfb5a994c7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:33.803041 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7034b90-effc-43f2-acdb-1ebfb5a994c7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:33.804816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7034b90-effc-43f2-acdb-1ebfb5a994c7 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:33.819796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-d7034b90-effc-43f2-acdb-1ebfb5a994c7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 14:58:33.819796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:33.819796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:33.819796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:33.819796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:33.819796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:33.819796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:33.819796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:33.819796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:33.819796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:33.819796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:33.819796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:33.819796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:33.819796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:33.819796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:33.819796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:33.821393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:33.821393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:33.821393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:33.821393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:33.821393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:33.821393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:33.821393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:33.821393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:33.821393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:33.821393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:33.821393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:33.821393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:33.821393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:33.821393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:33.821393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:33.821393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:33.821393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:33.821393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:33.822716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:33.822716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:33.822716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:33.822716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:33.822716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:33.822716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:33.822716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:33.822716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:33.822716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 14:58:33.822716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:33.822716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 332/658] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:58:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:33.822716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7034b90-effc-43f2-acdb-1ebfb5a994c7 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:33.836873 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb883ba-6846-40e5-8398-2f4912ce8542 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:33.896344 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d51a90-4db1-4dac-9244-45814dfef2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:33.897664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d51a90-4db1-4dac-9244-45814dfef2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:33.898026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d51a90-4db1-4dac-9244-45814dfef2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:33.898456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d51a90-4db1-4dac-9244-45814dfef2f5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:33.909582 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d51a90-4db1-4dac-9244-45814dfef2f5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:33.911118 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d51a90-4db1-4dac-9244-45814dfef2f5 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:33.928172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-67d51a90-4db1-4dac-9244-45814dfef2f5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 14:58:33.928172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:33.928172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:33.928172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:33.928172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:33.928172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:33.928172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:33.928172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:33.928172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:33.928172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:33.928172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:33.928172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:33.928172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:33.928172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:33.928172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:33.928172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:33.929856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:33.929856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:33.929856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:33.929856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:33.929856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:33.929856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:33.929856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:33.929856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:33.929856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:33.929856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:33.929856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:33.929856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:33.929856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:33.929856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:33.929856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:33.929856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:33.929856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:33.929856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:33.931041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:33.931041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:33.931041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:33.931041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:33.931041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:33.931041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:33.931041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:33.931041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:33.931041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 14:58:33.931041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:33.931041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 327/659] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:33.932386 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9f75b-55e4-4218-b019-a2ffc8d0e679 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:33.932587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9f75b-55e4-4218-b019-a2ffc8d0e679 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:33.932872 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9f75b-55e4-4218-b019-a2ffc8d0e679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:33.954706 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-8fd9f75b-55e4-4218-b019-a2ffc8d0e679 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:33.956212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-8fd9f75b-55e4-4218-b019-a2ffc8d0e679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:33.970886 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7034b90-effc-43f2-acdb-1ebfb5a994c7 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:34.008120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:34.022166 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae919bee-c86b-42d8-b0d6-522326763232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:34.022966 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae919bee-c86b-42d8-b0d6-522326763232 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:34.023247 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae919bee-c86b-42d8-b0d6-522326763232 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:34.023566 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae919bee-c86b-42d8-b0d6-522326763232 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:34.024430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae919bee-c86b-42d8-b0d6-522326763232 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:34.024896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae919bee-c86b-42d8-b0d6-522326763232 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:34.026573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae919bee-c86b-42d8-b0d6-522326763232 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:34.046911 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 333/660] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:34.046911 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae919bee-c86b-42d8-b0d6-522326763232 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:34.047988 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:34.063307 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 328/661] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:34.064089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9f75b-55e4-4218-b019-a2ffc8d0e679 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:34.196558 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae919bee-c86b-42d8-b0d6-522326763232 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:34.209539 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9f75b-55e4-4218-b019-a2ffc8d0e679 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:34.248764 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063b6a9-3a22-46e8-bf16-8895268d7438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:34.249462 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063b6a9-3a22-46e8-bf16-8895268d7438 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:34.249729 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063b6a9-3a22-46e8-bf16-8895268d7438 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:34.250004 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063b6a9-3a22-46e8-bf16-8895268d7438 None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:34.250801 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063b6a9-3a22-46e8-bf16-8895268d7438 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:34.251110 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063b6a9-3a22-46e8-bf16-8895268d7438 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:34.252596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063b6a9-3a22-46e8-bf16-8895268d7438 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:34.263371 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed81e5-fcd2-4909-9468-96a27e4e4a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:34.265714 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed81e5-fcd2-4909-9468-96a27e4e4a96 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:34.265714 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed81e5-fcd2-4909-9468-96a27e4e4a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:34.266044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed81e5-fcd2-4909-9468-96a27e4e4a96 None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:34.267240 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed81e5-fcd2-4909-9468-96a27e4e4a96 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:34.267602 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed81e5-fcd2-4909-9468-96a27e4e4a96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:34.270854 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed81e5-fcd2-4909-9468-96a27e4e4a96 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:34.279105 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 334/662] 10.222.0.93 () {56 vars in 1137 bytes} [Mon Aug 1 14:58:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:34.279632 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063b6a9-3a22-46e8-bf16-8895268d7438 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:34.301178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 329/663] 10.222.0.93 () {56 vars in 1139 bytes} [Mon Aug 1 14:58:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:34.302363 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed81e5-fcd2-4909-9468-96a27e4e4a96 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:34.344097 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:34.374234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:34.424869 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063b6a9-3a22-46e8-bf16-8895268d7438 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:34.443880 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed81e5-fcd2-4909-9468-96a27e4e4a96 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:34.474103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3713c-c7aa-49da-bdd4-3855784e3cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:34.474785 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3713c-c7aa-49da-bdd4-3855784e3cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:34.475066 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3713c-c7aa-49da-bdd4-3855784e3cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:34.475345 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3713c-c7aa-49da-bdd4-3855784e3cf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:34.483111 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3713c-c7aa-49da-bdd4-3855784e3cf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:34.484664 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3713c-c7aa-49da-bdd4-3855784e3cf2 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:34.493412 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab0fbc-f62a-43bf-b7b6-b93a6d86ae7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:34.494224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab0fbc-f62a-43bf-b7b6-b93a6d86ae7b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:34.494510 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab0fbc-f62a-43bf-b7b6-b93a6d86ae7b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:34.494785 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab0fbc-f62a-43bf-b7b6-b93a6d86ae7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:34.498361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-40a3713c-c7aa-49da-bdd4-3855784e3cf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:34.498361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:34.498361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:34.498361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:34.498361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:34.498361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:34.498361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:34.498361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:34.498361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:34.498361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:34.498361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:34.498361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:34.498361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:34.498361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:34.498361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:34.499312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:34.499312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:34.499312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:34.499312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:34.499312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:34.499312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:34.499312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:34.499312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:34.499312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:34.499312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:34.499312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:34.499312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:34.499312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:34.499312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:34.499312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:34.499312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:34.499312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:34.500396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:34.500396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:34.500396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:34.500396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:34.500396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:34.500396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:34.500396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:34.500396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:34.500396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 335/664] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:34.501400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3713c-c7aa-49da-bdd4-3855784e3cf2 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:34.516200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab0fbc-f62a-43bf-b7b6-b93a6d86ae7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:34.522332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-12ab0fbc-f62a-43bf-b7b6-b93a6d86ae7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:34.522332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:34.522332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:34.522332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:34.522332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:34.522332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:34.522332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:34.522332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:34.522332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:34.522332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:34.522332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:34.522332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:34.522332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:34.522332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:34.522332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:34.522332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:34.523595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:34.523595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:34.523595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:34.523595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:34.523595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:34.523595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:34.523595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:34.523595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:34.523595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:34.523595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:34.523595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:34.523595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:34.523595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:34.523595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:34.523595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:34.523595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:34.525102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:34.525102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:34.525102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:34.525102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:34.525102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:34.525102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:34.525102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:34.525102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:34.525102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:34.525102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:34.525102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:34.525102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:34.525102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 330/665] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:34.525102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab0fbc-f62a-43bf-b7b6-b93a6d86ae7b None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:34.640606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:34.646321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3713c-c7aa-49da-bdd4-3855784e3cf2 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:34.654721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab0fbc-f62a-43bf-b7b6-b93a6d86ae7b None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:34.670871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:34.692949 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb9a17-9f98-4804-9710-1eddae83e9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:34.693614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb9a17-9f98-4804-9710-1eddae83e9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:34.693889 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb9a17-9f98-4804-9710-1eddae83e9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:34.694154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb9a17-9f98-4804-9710-1eddae83e9a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:34.700725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed263efe-ceb5-48cc-adb4-f390f3a2d927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:34.701447 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed263efe-ceb5-48cc-adb4-f390f3a2d927 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:34.701791 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed263efe-ceb5-48cc-adb4-f390f3a2d927 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:34.702124 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed263efe-ceb5-48cc-adb4-f390f3a2d927 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:34.786874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb9a17-9f98-4804-9710-1eddae83e9a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:34.788346 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb9a17-9f98-4804-9710-1eddae83e9a5 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:34.792354 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed263efe-ceb5-48cc-adb4-f390f3a2d927 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:34.794844 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed263efe-ceb5-48cc-adb4-f390f3a2d927 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:34.935545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:34.944974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 336/666] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:58:34.946267 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb9a17-9f98-4804-9710-1eddae83e9a5 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:34.951445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 331/667] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:58:34.951445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed263efe-ceb5-48cc-adb4-f390f3a2d927 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:34.967020 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:35.088948 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed263efe-ceb5-48cc-adb4-f390f3a2d927 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:35.090402 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb9a17-9f98-4804-9710-1eddae83e9a5 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:35.134278 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ff83a-fbfc-45f4-9a5c-ecfe81d1fdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:35.135321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ff83a-fbfc-45f4-9a5c-ecfe81d1fdbf None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:35.135756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ff83a-fbfc-45f4-9a5c-ecfe81d1fdbf None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:35.136587 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ff83a-fbfc-45f4-9a5c-ecfe81d1fdbf None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:35.137643 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ff83a-fbfc-45f4-9a5c-ecfe81d1fdbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:35.138201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ff83a-fbfc-45f4-9a5c-ecfe81d1fdbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:35.140016 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ff83a-fbfc-45f4-9a5c-ecfe81d1fdbf None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:35.149163 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88970e31-3187-4cec-90e5-94b8a8d8177b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:35.150031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88970e31-3187-4cec-90e5-94b8a8d8177b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:35.150540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88970e31-3187-4cec-90e5-94b8a8d8177b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:35.151033 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88970e31-3187-4cec-90e5-94b8a8d8177b None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:35.152245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88970e31-3187-4cec-90e5-94b8a8d8177b None admin] RBAC: Adding query filter params (name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:35.152819 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88970e31-3187-4cec-90e5-94b8a8d8177b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:35.155440 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88970e31-3187-4cec-90e5-94b8a8d8177b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:35.160839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 337/668] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:35.161950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ff83a-fbfc-45f4-9a5c-ecfe81d1fdbf None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:35.171518 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 332/669] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 14:58:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:35.173246 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88970e31-3187-4cec-90e5-94b8a8d8177b None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:35.235738 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:35.275747 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:35.293614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ff83a-fbfc-45f4-9a5c-ecfe81d1fdbf None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:35.310023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88970e31-3187-4cec-90e5-94b8a8d8177b None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:35.341917 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7e1dd-1bec-49a6-bef9-1ff1163311e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:35.342686 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7e1dd-1bec-49a6-bef9-1ff1163311e9 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:35.343029 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7e1dd-1bec-49a6-bef9-1ff1163311e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:35.343379 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7e1dd-1bec-49a6-bef9-1ff1163311e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:35.355383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7e1dd-1bec-49a6-bef9-1ff1163311e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:35.356871 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7e1dd-1bec-49a6-bef9-1ff1163311e9 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:35.357813 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4153417-8c6e-45bd-ad33-86fba688bc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:35.358536 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4153417-8c6e-45bd-ad33-86fba688bc47 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:35.358896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4153417-8c6e-45bd-ad33-86fba688bc47 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:35.359248 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4153417-8c6e-45bd-ad33-86fba688bc47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:35.373266 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4153417-8c6e-45bd-ad33-86fba688bc47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:35.373986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-ddd7e1dd-1bec-49a6-bef9-1ff1163311e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:35.373986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:35.373986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:35.373986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:35.373986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:35.373986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:35.373986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:35.373986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:35.373986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:35.373986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:35.373986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:35.373986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:35.373986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:35.373986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:35.373986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:35.375233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:35.375233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:35.375233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:35.375233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:35.375233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:35.375233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:35.375233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:35.375233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:35.375233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:35.375233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:35.375233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:35.375233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:35.375233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:35.375233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:35.375233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:35.375233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:35.375233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:35.382203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:35.382203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:35.382203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:35.382203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:35.382203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 338/670] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:35.382588 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4153417-8c6e-45bd-ad33-86fba688bc47 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:35.382678 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7e1dd-1bec-49a6-bef9-1ff1163311e9 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:35.402258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-d4153417-8c6e-45bd-ad33-86fba688bc47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:35.402258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:35.402258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:35.402258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:35.402258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:35.402258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:35.402258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:35.402258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:35.402258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:35.402258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:35.402258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:35.402258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:35.402258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:35.402258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:35.402258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:35.403420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:35.403420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:35.403420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:35.403420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:35.403420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:35.403420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:35.403420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:35.403420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:35.403420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:35.403420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:35.403420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:35.403420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:35.403420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:35.403420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:35.403420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:35.403420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:35.403420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:35.404499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:35.404499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:35.404499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:35.404499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:35.409534 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 333/671] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:35.410495 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4153417-8c6e-45bd-ad33-86fba688bc47 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:35.533909 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7e1dd-1bec-49a6-bef9-1ff1163311e9 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:35.550728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:35.561054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4153417-8c6e-45bd-ad33-86fba688bc47 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:35.588919 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb1542-b6c3-4072-9740-06f4d18805c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:35.589680 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb1542-b6c3-4072-9740-06f4d18805c9 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:35.590026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb1542-b6c3-4072-9740-06f4d18805c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:35.590382 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb1542-b6c3-4072-9740-06f4d18805c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:35.591265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb1542-b6c3-4072-9740-06f4d18805c9 None admin] RBAC: Adding query filter params (role.id=6cff92a2d53f4855b38a9d9f11aa19bc, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=3d9be74621d14c94b95ae94947e5abbd) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:35.591657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb1542-b6c3-4072-9740-06f4d18805c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:35.593276 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb1542-b6c3-4072-9740-06f4d18805c9 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:35.594253 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:35.608491 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 339/672] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:58:35 2022] GET /identity/v3/role_assignments?user.id=3d9be74621d14c94b95ae94947e5abbd&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=6cff92a2d53f4855b38a9d9f11aa19bc => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:35.608491 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb1542-b6c3-4072-9740-06f4d18805c9 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:35.617323 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d9b69-8bbc-4b05-aa46-dd77d2338b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:35.618390 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d9b69-8bbc-4b05-aa46-dd77d2338b89 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:35.618920 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d9b69-8bbc-4b05-aa46-dd77d2338b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:35.619206 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d9b69-8bbc-4b05-aa46-dd77d2338b89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:35.637650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d9b69-8bbc-4b05-aa46-dd77d2338b89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:35.640530 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d9b69-8bbc-4b05-aa46-dd77d2338b89 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:35.668159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-ce5d9b69-8bbc-4b05-aa46-dd77d2338b89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:35.668159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:35.668159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:35.668159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:35.668159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:35.668159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:35.668159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:35.668159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:35.668159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:35.668159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:35.668159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:35.668159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:35.668159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:35.668159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:35.668159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:35.669430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:35.669430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:35.669430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:35.669430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:35.669430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:35.669430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:35.669430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:35.669430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:35.669430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:35.669430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:35.669430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:35.669430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:35.669430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:35.669430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:35.669430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:35.669430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:35.669430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:35.670385 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:35.670385 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:35.670385 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:35.670385 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:35.670385 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 334/673] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:35.671334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d9b69-8bbc-4b05-aa46-dd77d2338b89 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:35.792422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb1542-b6c3-4072-9740-06f4d18805c9 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:35.836939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ba2e3-8236-4793-9c03-440bf5c85add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:35.837697 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ba2e3-8236-4793-9c03-440bf5c85add None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:35.838012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ba2e3-8236-4793-9c03-440bf5c85add None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:35.838263 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ba2e3-8236-4793-9c03-440bf5c85add None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:35.838967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ba2e3-8236-4793-9c03-440bf5c85add None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:35.839268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ba2e3-8236-4793-9c03-440bf5c85add None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:35.840915 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ba2e3-8236-4793-9c03-440bf5c85add None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:35.855786 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 340/674] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:35.856971 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d9b69-8bbc-4b05-aa46-dd77d2338b89 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:35.858024 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ba2e3-8236-4793-9c03-440bf5c85add None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:35.905765 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:35.914769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fc4f8-fd12-4ac1-bc2a-918b09897aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:35.915802 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fc4f8-fd12-4ac1-bc2a-918b09897aff None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:35.916298 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fc4f8-fd12-4ac1-bc2a-918b09897aff None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:35.916812 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fc4f8-fd12-4ac1-bc2a-918b09897aff None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:35.917712 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fc4f8-fd12-4ac1-bc2a-918b09897aff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:35.918208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fc4f8-fd12-4ac1-bc2a-918b09897aff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:35.919828 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fc4f8-fd12-4ac1-bc2a-918b09897aff None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:35.935278 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 335/675] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:35.936672 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fc4f8-fd12-4ac1-bc2a-918b09897aff None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:35.981756 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:36.001243 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ba2e3-8236-4793-9c03-440bf5c85add None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:36.042979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5b332-86c0-4ed6-8c80-0fc707d559c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:36.043453 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5b332-86c0-4ed6-8c80-0fc707d559c9 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:36.043794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5b332-86c0-4ed6-8c80-0fc707d559c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:36.044075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5b332-86c0-4ed6-8c80-0fc707d559c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:36.044924 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5b332-86c0-4ed6-8c80-0fc707d559c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:36.045237 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5b332-86c0-4ed6-8c80-0fc707d559c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:36.046648 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5b332-86c0-4ed6-8c80-0fc707d559c9 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:36.050612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fc4f8-fd12-4ac1-bc2a-918b09897aff None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:36.060947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 341/676] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:36.062294 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f2a55-e2df-44eb-83e5-656b79bbd5f7 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:36.062607 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f2a55-e2df-44eb-83e5-656b79bbd5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:36.062950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f2a55-e2df-44eb-83e5-656b79bbd5f7 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:36.068892 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 342/677] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:36.068892 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f2a55-e2df-44eb-83e5-656b79bbd5f7 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:36.101414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63920cd-c38f-4212-aa06-3eeceb35a19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:36.101804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63920cd-c38f-4212-aa06-3eeceb35a19d None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:36.102547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63920cd-c38f-4212-aa06-3eeceb35a19d None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:36.102830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63920cd-c38f-4212-aa06-3eeceb35a19d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:36.111472 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63920cd-c38f-4212-aa06-3eeceb35a19d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:36.113697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63920cd-c38f-4212-aa06-3eeceb35a19d None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:36.130315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-d63920cd-c38f-4212-aa06-3eeceb35a19d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:36.130315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:36.130315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:36.130315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:36.130315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:36.130315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:36.130315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:36.130315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:36.130315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:36.130315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:36.130315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:36.130315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:36.130315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:36.130315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:36.130315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:36.132096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:36.132096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:36.132096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:36.132096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:36.132096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:36.132096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:36.132096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:36.132096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:36.132096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:36.132096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:36.132096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:36.132096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:36.132096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:36.132096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:36.132096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:36.132096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:36.132096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:36.134648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:36.134648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:36.134648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:36.134648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:36.134648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:36.134648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:36.134648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:36.134648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:36.134648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 336/678] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:36.134648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63920cd-c38f-4212-aa06-3eeceb35a19d None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:36.249314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:36.252990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:36.259625 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f2a55-e2df-44eb-83e5-656b79bbd5f7 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:36.260650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63920cd-c38f-4212-aa06-3eeceb35a19d None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:36.297988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c66e5-226b-46db-b022-d9f418168092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:36.298565 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c66e5-226b-46db-b022-d9f418168092 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:36.298861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c66e5-226b-46db-b022-d9f418168092 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:36.299109 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c66e5-226b-46db-b022-d9f418168092 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:36.307791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c66e5-226b-46db-b022-d9f418168092 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:36.309360 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c66e5-226b-46db-b022-d9f418168092 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:36.322747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-516c66e5-226b-46db-b022-d9f418168092 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:36.322747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:36.322747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:36.322747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:36.322747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:36.322747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:36.322747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:36.322747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:36.322747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:36.322747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:36.322747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:36.322747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:36.322747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:36.322747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:36.322747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:36.323856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:36.323856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:36.323856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:36.323856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:36.323856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:36.323856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:36.323856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:36.323856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:36.323856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:36.323856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:36.323856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:36.323856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:36.323856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:36.323856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:36.323856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:36.323856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:36.323856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:36.325190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:36.325190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:36.325190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:36.325190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:36.325190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:36.325190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:36.325190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:36.325190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:36.328842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 343/679] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:36.330632 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4fb282-c859-4ea7-bde2-74527148245b None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:36.330926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4fb282-c859-4ea7-bde2-74527148245b None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:36.333028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4fb282-c859-4ea7-bde2-74527148245b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:36.351004 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-5d4fb282-c859-4ea7-bde2-74527148245b None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:36.352235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5d4fb282-c859-4ea7-bde2-74527148245b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:36.440487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 344/680] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:36.440487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4fb282-c859-4ea7-bde2-74527148245b None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:36.483114 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bbace-e910-42ba-a8c0-e9c8a70a6f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:36.483812 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bbace-e910-42ba-a8c0-e9c8a70a6f93 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:36.484092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bbace-e910-42ba-a8c0-e9c8a70a6f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:36.484367 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bbace-e910-42ba-a8c0-e9c8a70a6f93 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:36.497130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bbace-e910-42ba-a8c0-e9c8a70a6f93 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:36.498715 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bbace-e910-42ba-a8c0-e9c8a70a6f93 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:36.512317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-b61bbace-e910-42ba-a8c0-e9c8a70a6f93 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:58:36.512317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:36.512317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:36.512317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:36.512317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:36.512317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:36.512317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:36.512317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:36.512317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:36.512317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:36.512317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:36.512317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:36.512317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:36.512317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:36.512317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:36.512317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:36.513378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:36.513378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:36.513378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:36.513378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:36.513378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:36.513378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:36.513378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:36.513378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:36.513378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:36.513378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:36.513378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:36.513378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:36.513378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:36.513378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:36.513378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:36.513378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:36.513378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:36.513378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:36.514358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:36.514358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:36.514358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:36.514358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:36.514358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:36.514358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:36.514358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:36.514358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:36.514358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:58:36.514358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:36.516606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 337/681] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:58:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:36.517941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bbace-e910-42ba-a8c0-e9c8a70a6f93 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:36.588199 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:36.613330 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4fb282-c859-4ea7-bde2-74527148245b None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:36.631657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bbace-e910-42ba-a8c0-e9c8a70a6f93 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:36.670117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f12ee-eb58-41cf-a0a9-3c43dad186cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:36.670735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f12ee-eb58-41cf-a0a9-3c43dad186cb None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:36.671018 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f12ee-eb58-41cf-a0a9-3c43dad186cb None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:36.671289 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f12ee-eb58-41cf-a0a9-3c43dad186cb None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:36.672034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f12ee-eb58-41cf-a0a9-3c43dad186cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:36.672335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f12ee-eb58-41cf-a0a9-3c43dad186cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:36.673582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79bd9f-5cb2-468c-834a-286e6ae2b268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:36.673867 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f12ee-eb58-41cf-a0a9-3c43dad186cb None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:36.674594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79bd9f-5cb2-468c-834a-286e6ae2b268 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:36.675065 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79bd9f-5cb2-468c-834a-286e6ae2b268 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:36.675529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79bd9f-5cb2-468c-834a-286e6ae2b268 None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:36.677516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79bd9f-5cb2-468c-834a-286e6ae2b268 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:36.678004 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79bd9f-5cb2-468c-834a-286e6ae2b268 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:36.680737 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79bd9f-5cb2-468c-834a-286e6ae2b268 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:36.692431 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 338/682] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:36.692431 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f12ee-eb58-41cf-a0a9-3c43dad186cb None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:36.708141 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 345/683] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:36.708141 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79bd9f-5cb2-468c-834a-286e6ae2b268 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:36.722924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:36.834319 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f12ee-eb58-41cf-a0a9-3c43dad186cb None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:36.871385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79bd9f-5cb2-468c-834a-286e6ae2b268 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:36.882219 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6abaa-a106-4e40-9613-0d6e77904871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:36.882893 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6abaa-a106-4e40-9613-0d6e77904871 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:36.883263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6abaa-a106-4e40-9613-0d6e77904871 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:36.883507 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6abaa-a106-4e40-9613-0d6e77904871 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:36.884123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6abaa-a106-4e40-9613-0d6e77904871 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:36.885596 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6abaa-a106-4e40-9613-0d6e77904871 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:36.898953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-dbe6abaa-a106-4e40-9613-0d6e77904871 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 14:58:36.898953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:36.898953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:58:36.898953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:58:36.898953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:58:36.898953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:58:36.898953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:58:36.898953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:58:36.898953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 14:58:36.898953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:36.898953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:58:36.898953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:36.898953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:36.898953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:36.898953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:36.898953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:36.900157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:36.900157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:36.900157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:36.900157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:36.900157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:36.900157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:36.900157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:36.900157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:58:36.900157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:58:36.900157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:58:36.900157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:58:36.900157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:36.900157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:36.900157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:36.900157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:36.900157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:36.901271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:36.901271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:36.901271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:36.901271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:36.901271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:36.901271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:36.901271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:36.901271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:36.901271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:36.901271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:36.901271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:36.901271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:58:36.901271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:58:36.901271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 14:58:36.901271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:36.901271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 339/684] 10.222.0.93 () {56 vars in 1129 bytes} [Mon Aug 1 14:58:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:36.903015 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6abaa-a106-4e40-9613-0d6e77904871 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:36.923754 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59d8e8-e452-487e-b2d6-061681e7c54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:36.924304 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59d8e8-e452-487e-b2d6-061681e7c54b None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:36.924621 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59d8e8-e452-487e-b2d6-061681e7c54b None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:36.924897 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59d8e8-e452-487e-b2d6-061681e7c54b None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:36.925733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59d8e8-e452-487e-b2d6-061681e7c54b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:36.926139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59d8e8-e452-487e-b2d6-061681e7c54b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:36.928187 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59d8e8-e452-487e-b2d6-061681e7c54b None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:36.948777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 346/685] 10.222.0.93 () {56 vars in 1135 bytes} [Mon Aug 1 14:58:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:36.949662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59d8e8-e452-487e-b2d6-061681e7c54b None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:36.954867 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:37.016195 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:37.041889 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6abaa-a106-4e40-9613-0d6e77904871 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:37.081573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59d8e8-e452-487e-b2d6-061681e7c54b None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:37.082170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb4b88-fca2-4fde-9c51-9244e66648e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:37.082816 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb4b88-fca2-4fde-9c51-9244e66648e9 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:37.083627 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb4b88-fca2-4fde-9c51-9244e66648e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:37.083901 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb4b88-fca2-4fde-9c51-9244e66648e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:37.084725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb4b88-fca2-4fde-9c51-9244e66648e9 None admin] RBAC: Adding query filter params (role.id=6cff92a2d53f4855b38a9d9f11aa19bc, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=4eba5f64833b482d81c53721c8442970) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:37.085036 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb4b88-fca2-4fde-9c51-9244e66648e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:37.086485 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb4b88-fca2-4fde-9c51-9244e66648e9 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:37.092791 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 340/686] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:58:36 2022] GET /identity/v3/role_assignments?user.id=4eba5f64833b482d81c53721c8442970&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=6cff92a2d53f4855b38a9d9f11aa19bc => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:37.093634 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb4b88-fca2-4fde-9c51-9244e66648e9 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:37.152754 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9b585-4b6f-42be-8a64-5d376ebd67ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:37.154422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9b585-4b6f-42be-8a64-5d376ebd67ab None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:37.155211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9b585-4b6f-42be-8a64-5d376ebd67ab None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:37.156165 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9b585-4b6f-42be-8a64-5d376ebd67ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:37.159380 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9b585-4b6f-42be-8a64-5d376ebd67ab None admin] RBAC: Adding query filter params (role.id=6cff92a2d53f4855b38a9d9f11aa19bc, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=4461a389da5848f5afd9854758707829) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:37.160600 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9b585-4b6f-42be-8a64-5d376ebd67ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:37.164418 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9b585-4b6f-42be-8a64-5d376ebd67ab None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:37.182864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 347/687] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:58:36 2022] GET /identity/v3/role_assignments?user.id=4461a389da5848f5afd9854758707829&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=6cff92a2d53f4855b38a9d9f11aa19bc => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:37.184877 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9b585-4b6f-42be-8a64-5d376ebd67ab None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:37.318384 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb4b88-fca2-4fde-9c51-9244e66648e9 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:37.347320 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:37.392062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d587c-a1de-4b1a-a4ab-dc915c1abe74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:37.392904 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d587c-a1de-4b1a-a4ab-dc915c1abe74 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:37.393712 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d587c-a1de-4b1a-a4ab-dc915c1abe74 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:37.393999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d587c-a1de-4b1a-a4ab-dc915c1abe74 None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:37.395008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d587c-a1de-4b1a-a4ab-dc915c1abe74 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:37.395319 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d587c-a1de-4b1a-a4ab-dc915c1abe74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:37.397533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d587c-a1de-4b1a-a4ab-dc915c1abe74 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:37.401915 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9b585-4b6f-42be-8a64-5d376ebd67ab None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:37.403909 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 341/688] 10.222.0.93 () {56 vars in 1124 bytes} [Mon Aug 1 14:58:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:37.409977 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d587c-a1de-4b1a-a4ab-dc915c1abe74 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:37.440895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:37.453860 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2925e7c-6a9e-4019-a0e8-e9b285044de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:37.454564 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2925e7c-6a9e-4019-a0e8-e9b285044de4 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:37.454845 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2925e7c-6a9e-4019-a0e8-e9b285044de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:37.455222 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2925e7c-6a9e-4019-a0e8-e9b285044de4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:37.566404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d587c-a1de-4b1a-a4ab-dc915c1abe74 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:37.603242 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2925e7c-6a9e-4019-a0e8-e9b285044de4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:37.605632 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2925e7c-6a9e-4019-a0e8-e9b285044de4 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:37.616148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58aa5a0-7b6a-4828-be42-2681642d395a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:37.617028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58aa5a0-7b6a-4828-be42-2681642d395a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:37.617363 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58aa5a0-7b6a-4828-be42-2681642d395a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:37.617643 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58aa5a0-7b6a-4828-be42-2681642d395a None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:37.618491 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58aa5a0-7b6a-4828-be42-2681642d395a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:37.619111 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58aa5a0-7b6a-4828-be42-2681642d395a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:37.621127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58aa5a0-7b6a-4828-be42-2681642d395a None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:37.628974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 342/689] 10.222.0.93 () {56 vars in 1124 bytes} [Mon Aug 1 14:58:37 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:58:37.690075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:37.735792 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 348/690] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:58:37.742472 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58aa5a0-7b6a-4828-be42-2681642d395a None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:37.847323 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:37.884184 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58aa5a0-7b6a-4828-be42-2681642d395a None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:37.925632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775dd737-2f7a-4432-93be-420975b31918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:37.926606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dd737-2f7a-4432-93be-420975b31918 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:37.927070 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dd737-2f7a-4432-93be-420975b31918 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:37.927500 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dd737-2f7a-4432-93be-420975b31918 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:37.945850 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775dd737-2f7a-4432-93be-420975b31918 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:37.947667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775dd737-2f7a-4432-93be-420975b31918 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:37.965753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-775dd737-2f7a-4432-93be-420975b31918 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:37.965753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:37.965753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:37.965753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:37.965753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:37.965753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:37.965753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:37.965753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:37.965753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:37.965753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:37.965753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:37.965753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:37.965753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:37.965753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:37.965753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:37.966982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:37.966982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:37.966982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:37.966982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:37.966982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:37.966982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:37.966982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:37.966982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:37.966982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:37.966982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:37.966982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:37.966982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:37.966982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:37.966982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:37.966982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:37.966982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:37.966982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:37.968031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:37.968031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:37.968031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:37.968031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:37.970062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 343/691] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:37.978725 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2925e7c-6a9e-4019-a0e8-e9b285044de4 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:38.144362 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2925e7c-6a9e-4019-a0e8-e9b285044de4 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:38.188622 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd51a1-bf7f-4ba4-8dbe-dd2096955180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:38.189443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd51a1-bf7f-4ba4-8dbe-dd2096955180 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:38.189723 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd51a1-bf7f-4ba4-8dbe-dd2096955180 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:38.190014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd51a1-bf7f-4ba4-8dbe-dd2096955180 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:38.190732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd51a1-bf7f-4ba4-8dbe-dd2096955180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:38.191061 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd51a1-bf7f-4ba4-8dbe-dd2096955180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:38.192988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd51a1-bf7f-4ba4-8dbe-dd2096955180 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:38.220322 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523223f-3466-4027-ba7d-0fdc61675e4e None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:38.220640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523223f-3466-4027-ba7d-0fdc61675e4e None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:38.220936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523223f-3466-4027-ba7d-0fdc61675e4e None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:38.222581 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 344/692] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:38.223097 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 349/693] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:38.228439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e601b-5ab2-4fbf-9fb3-6f6c11428e11 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:38.228764 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e601b-5ab2-4fbf-9fb3-6f6c11428e11 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:38.229016 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e601b-5ab2-4fbf-9fb3-6f6c11428e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:38.233068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd51a1-bf7f-4ba4-8dbe-dd2096955180 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:38.289000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-e83e601b-5ab2-4fbf-9fb3-6f6c11428e11 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:38.291024 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e83e601b-5ab2-4fbf-9fb3-6f6c11428e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:38.311730 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:38.421383 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 345/694] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:38.438422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd51a1-bf7f-4ba4-8dbe-dd2096955180 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:38.446953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e601b-5ab2-4fbf-9fb3-6f6c11428e11 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:38.524837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:38.553248 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f28bdf-b595-436a-b47c-1866b9d6f8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:38.553913 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f28bdf-b595-436a-b47c-1866b9d6f8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:38.554196 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f28bdf-b595-436a-b47c-1866b9d6f8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:38.554530 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f28bdf-b595-436a-b47c-1866b9d6f8d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:38.585988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f28bdf-b595-436a-b47c-1866b9d6f8d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:38.588494 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f28bdf-b595-436a-b47c-1866b9d6f8d1 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:38.593324 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e601b-5ab2-4fbf-9fb3-6f6c11428e11 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:38.623089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-61f28bdf-b595-436a-b47c-1866b9d6f8d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:38.623089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:38.623089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:38.623089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:38.623089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:38.623089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:38.623089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:38.623089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:38.623089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:38.623089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:38.623089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:38.623089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:38.623089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:38.623089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:38.623089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:38.624062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:38.624062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:38.624062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:38.624062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:38.624062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:38.624062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:38.624062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:38.624062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:38.624062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:38.624062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:38.624062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:38.624062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:38.624062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:38.624062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:38.624062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:38.624062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:38.624062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:38.625374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:38.625374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:38.625374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:38.625374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:38.625374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:38.625374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:38.625374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:38.625374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:38.625374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 350/695] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:38.646086 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f28bdf-b595-436a-b47c-1866b9d6f8d1 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:38.669403 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7facf1d-84bf-4b41-b92c-2644c0d276c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:38.671442 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7facf1d-84bf-4b41-b92c-2644c0d276c2 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:38.672098 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7facf1d-84bf-4b41-b92c-2644c0d276c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:38.672721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7facf1d-84bf-4b41-b92c-2644c0d276c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:38.694208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7facf1d-84bf-4b41-b92c-2644c0d276c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:38.699781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-f7facf1d-84bf-4b41-b92c-2644c0d276c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:58:38.699781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:38.699781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:38.699781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:38.699781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:38.699781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:38.699781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:38.699781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:38.699781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:38.699781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:38.699781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:38.699781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:38.699781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:38.699781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:38.699781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:38.699781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:38.701837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:38.701837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:38.701837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:38.701837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:38.701837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:38.701837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:38.701837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:38.701837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:38.701837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:38.701837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:38.701837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:38.701837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:38.701837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:38.701837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:38.701837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:38.701837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:38.704005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:38.704005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:38.704005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:38.704005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:38.704005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:38.704005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:38.704005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:38.704005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:38.704005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:38.704005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:38.704005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:58:38.704005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:38.704005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 346/696] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:58:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:58:38.709487 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7facf1d-84bf-4b41-b92c-2644c0d276c2 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:38.831989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7facf1d-84bf-4b41-b92c-2644c0d276c2 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:38.864645 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:38.894890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:38.906650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de8e77-c778-4ba4-852b-324045a58852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:38.907281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de8e77-c778-4ba4-852b-324045a58852 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:38.907566 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de8e77-c778-4ba4-852b-324045a58852 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:38.907841 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de8e77-c778-4ba4-852b-324045a58852 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:38.908587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de8e77-c778-4ba4-852b-324045a58852 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:38.908893 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de8e77-c778-4ba4-852b-324045a58852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:38.910279 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de8e77-c778-4ba4-852b-324045a58852 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:38.916317 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f28bdf-b595-436a-b47c-1866b9d6f8d1 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:38.939192 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 347/697] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:58:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:38.949526 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de8e77-c778-4ba4-852b-324045a58852 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:38.966321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a5f7c-f152-42ce-988b-3229621d2e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:38.967720 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a5f7c-f152-42ce-988b-3229621d2e14 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:38.968000 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a5f7c-f152-42ce-988b-3229621d2e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:38.968374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a5f7c-f152-42ce-988b-3229621d2e14 None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:38.972870 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a5f7c-f152-42ce-988b-3229621d2e14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:38.973342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a5f7c-f152-42ce-988b-3229621d2e14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:38.979104 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a5f7c-f152-42ce-988b-3229621d2e14 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:39.020423 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 351/698] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:39.040728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a5f7c-f152-42ce-988b-3229621d2e14 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:39.143837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de8e77-c778-4ba4-852b-324045a58852 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:39.194696 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c287c-6821-43db-b266-37e5cbcc7e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:39.196073 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c287c-6821-43db-b266-37e5cbcc7e74 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:39.196355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c287c-6821-43db-b266-37e5cbcc7e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:39.196678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c287c-6821-43db-b266-37e5cbcc7e74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:39.209568 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c287c-6821-43db-b266-37e5cbcc7e74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:39.211259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c287c-6821-43db-b266-37e5cbcc7e74 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:39.231153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-eb4c287c-6821-43db-b266-37e5cbcc7e74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:39.231153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:39.231153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:39.231153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:39.231153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:39.231153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:39.231153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:39.231153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:39.231153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:39.231153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:39.231153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:39.231153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:39.231153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:39.231153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:39.231153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:39.233419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:39.233419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:39.233419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:39.233419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:39.233419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:39.233419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:39.233419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:39.233419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:39.233419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:39.233419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:39.233419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:39.233419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:39.233419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:39.233419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:39.233419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:39.233419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:39.233419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:39.235687 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:39.235687 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:39.235687 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:39.235687 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:39.235687 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 348/699] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:39.242298 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c287c-6821-43db-b266-37e5cbcc7e74 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:39.292886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:39.399854 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a5f7c-f152-42ce-988b-3229621d2e14 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:39.422783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c287c-6821-43db-b266-37e5cbcc7e74 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:39.440968 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b40a31-210c-4fca-a4f5-148b461725ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:39.441693 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b40a31-210c-4fca-a4f5-148b461725ba None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:39.441886 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b40a31-210c-4fca-a4f5-148b461725ba None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:39.442188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b40a31-210c-4fca-a4f5-148b461725ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:39.443071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b40a31-210c-4fca-a4f5-148b461725ba None admin] RBAC: Adding query filter params (role.id=6cff92a2d53f4855b38a9d9f11aa19bc, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=4df37b02152c48509243b73ebbf49d8d) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:39.443387 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b40a31-210c-4fca-a4f5-148b461725ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:39.448493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b40a31-210c-4fca-a4f5-148b461725ba None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:39.453554 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 352/700] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:58:39 2022] GET /identity/v3/role_assignments?user.id=4df37b02152c48509243b73ebbf49d8d&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=6cff92a2d53f4855b38a9d9f11aa19bc => generated 608 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:58:39.534261 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa5ff3-fe30-46fa-8623-87a2f52370b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:39.534912 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa5ff3-fe30-46fa-8623-87a2f52370b1 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:39.535192 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa5ff3-fe30-46fa-8623-87a2f52370b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:39.535472 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa5ff3-fe30-46fa-8623-87a2f52370b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:39.536849 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa5ff3-fe30-46fa-8623-87a2f52370b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:39.537155 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa5ff3-fe30-46fa-8623-87a2f52370b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:39.538801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa5ff3-fe30-46fa-8623-87a2f52370b1 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:39.566676 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 349/701] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:58:39.572596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b40a31-210c-4fca-a4f5-148b461725ba None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:39.604622 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:39.732773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b40a31-210c-4fca-a4f5-148b461725ba None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:39.775824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479a37b-1295-4607-9e25-ef9b20327980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:39.776587 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479a37b-1295-4607-9e25-ef9b20327980 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:39.776871 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479a37b-1295-4607-9e25-ef9b20327980 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:39.777166 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479a37b-1295-4607-9e25-ef9b20327980 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:39.787229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479a37b-1295-4607-9e25-ef9b20327980 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:39.788801 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479a37b-1295-4607-9e25-ef9b20327980 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:39.808443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-1479a37b-1295-4607-9e25-ef9b20327980 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:58:39.808443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:39.808443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:39.808443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:39.808443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:39.808443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:39.808443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:39.808443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:39.808443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:39.808443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:39.808443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:39.808443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:39.808443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:39.808443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:39.808443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:39.808443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:39.811190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:39.811190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:39.811190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:39.811190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:39.811190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:39.811190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:39.811190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:39.811190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:39.811190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:39.811190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:39.811190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:39.811190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:39.811190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:39.811190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:39.811190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:39.811190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:39.811190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:39.811190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:39.816583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:39.816583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:39.816583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:39.816583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:39.816583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:39.816583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:39.816583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:39.816583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:39.816583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:58:39.816583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:39.816583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 353/702] 10.222.0.93 () {56 vars in 1093 bytes} [Mon Aug 1 14:58:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:39.818979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa5ff3-fe30-46fa-8623-87a2f52370b1 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:39.996023 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf82474-0773-4ac4-b263-fb3be0d25f58 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:39.996202 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf82474-0773-4ac4-b263-fb3be0d25f58 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:39.997709 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf82474-0773-4ac4-b263-fb3be0d25f58 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:40.000802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 354/703] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:40.010688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b4447-7f81-41cb-8fce-e134012022d2 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:40.011063 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b4447-7f81-41cb-8fce-e134012022d2 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:40.011357 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b4447-7f81-41cb-8fce-e134012022d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:40.014418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:40.032276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa5ff3-fe30-46fa-8623-87a2f52370b1 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:40.048122 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-b66b4447-7f81-41cb-8fce-e134012022d2 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:40.050595 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b66b4447-7f81-41cb-8fce-e134012022d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:40.076197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5d542-2464-4667-8c35-1741337ea938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:40.077753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5d542-2464-4667-8c35-1741337ea938 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:40.078521 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5d542-2464-4667-8c35-1741337ea938 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:40.079303 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5d542-2464-4667-8c35-1741337ea938 None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:40.081256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5d542-2464-4667-8c35-1741337ea938 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:40.082016 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5d542-2464-4667-8c35-1741337ea938 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:40.085646 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5d542-2464-4667-8c35-1741337ea938 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:40.128429 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 350/704] 10.222.0.93 () {56 vars in 1133 bytes} [Mon Aug 1 14:58:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:40.136770 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5d542-2464-4667-8c35-1741337ea938 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:40.201944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 355/705] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:58:40.231208 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b4447-7f81-41cb-8fce-e134012022d2 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:40.233965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:40.291910 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5d542-2464-4667-8c35-1741337ea938 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:40.350315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42128b4-3bf3-43fb-a3f3-bcb2ed620f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:40.350971 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42128b4-3bf3-43fb-a3f3-bcb2ed620f2f None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:40.351259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42128b4-3bf3-43fb-a3f3-bcb2ed620f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:40.351569 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42128b4-3bf3-43fb-a3f3-bcb2ed620f2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:40.394122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:40.446396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b4447-7f81-41cb-8fce-e134012022d2 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:40.470914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42128b4-3bf3-43fb-a3f3-bcb2ed620f2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:40.472589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42128b4-3bf3-43fb-a3f3-bcb2ed620f2f None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:40.494316 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496a679-47d0-4a6c-a260-bad5d9805db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:40.495373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496a679-47d0-4a6c-a260-bad5d9805db9 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:40.495935 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496a679-47d0-4a6c-a260-bad5d9805db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:40.496206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496a679-47d0-4a6c-a260-bad5d9805db9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:40.496894 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496a679-47d0-4a6c-a260-bad5d9805db9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:40.498340 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496a679-47d0-4a6c-a260-bad5d9805db9 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:40.514653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-3496a679-47d0-4a6c-a260-bad5d9805db9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 14:58:40.514653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:40.514653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:58:40.514653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:58:40.514653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:58:40.514653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:58:40.514653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:58:40.514653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:58:40.514653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 14:58:40.514653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:40.514653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:58:40.514653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:40.514653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:40.514653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:40.514653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:40.514653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:40.514653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:40.522210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:40.522210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:40.522210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:40.522210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:40.522210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:40.522210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:40.522210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:58:40.522210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:58:40.522210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:58:40.522210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:58:40.522210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:40.522210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:40.522210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:40.522210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:40.522210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:40.522210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:40.526487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:40.526487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:40.526487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:40.526487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:40.526487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:40.526487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:40.526487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:40.526487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:40.526487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:40.526487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:40.526487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:58:40.526487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:58:40.526487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 14:58:40.526487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:40.526487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 356/706] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:40 2022] GET /identity/v3/services/network => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:40.528359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cf631-55f9-4c4e-b6e6-a1fb30c3f5bb None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:40.528359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cf631-55f9-4c4e-b6e6-a1fb30c3f5bb None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:40.528359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cf631-55f9-4c4e-b6e6-a1fb30c3f5bb None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:40.528359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 357/707] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:40.528359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc62e4f-df35-4b70-95da-146c020209f5 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:40.528359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc62e4f-df35-4b70-95da-146c020209f5 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:40.528826 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc62e4f-df35-4b70-95da-146c020209f5 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:40.528826 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 358/708] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:40.529236 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ab2b4-7859-4a94-b035-814bcdb399ba None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:40.529525 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ab2b4-7859-4a94-b035-814bcdb399ba None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:40.529799 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ab2b4-7859-4a94-b035-814bcdb399ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:40.549560 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-375ab2b4-7859-4a94-b035-814bcdb399ba None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:40.550696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-375ab2b4-7859-4a94-b035-814bcdb399ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:40.627866 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:40.644246 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 359/709] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:40.644246 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ab2b4-7859-4a94-b035-814bcdb399ba None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:40.644522 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 351/710] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:58:40.647464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9a50d-ae2b-4709-871e-bd0c018a4529 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:40.648491 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9a50d-ae2b-4709-871e-bd0c018a4529 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:40.648491 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9a50d-ae2b-4709-871e-bd0c018a4529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:40.674153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-45b9a50d-ae2b-4709-871e-bd0c018a4529 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:40.675362 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-45b9a50d-ae2b-4709-871e-bd0c018a4529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:40.729670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:40.764738 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 352/711] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:40.764738 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9a50d-ae2b-4709-871e-bd0c018a4529 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:40.775050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ab2b4-7859-4a94-b035-814bcdb399ba None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:40.813693 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55437c21-bd38-4668-870f-0da89f4364c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:40.814233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55437c21-bd38-4668-870f-0da89f4364c9 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:40.814515 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55437c21-bd38-4668-870f-0da89f4364c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:40.814792 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55437c21-bd38-4668-870f-0da89f4364c9 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:40.815468 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55437c21-bd38-4668-870f-0da89f4364c9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:40.815761 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55437c21-bd38-4668-870f-0da89f4364c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:40.817839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55437c21-bd38-4668-870f-0da89f4364c9 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:40.822637 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 360/712] 10.222.0.93 () {56 vars in 1112 bytes} [Mon Aug 1 14:58:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:40.823433 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55437c21-bd38-4668-870f-0da89f4364c9 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:40.896237 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:40.905571 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9a50d-ae2b-4709-871e-bd0c018a4529 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:40.941130 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55437c21-bd38-4668-870f-0da89f4364c9 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:40.944231 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7e7c2-eb9f-4c0e-bb60-59474603862a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:40.944917 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7e7c2-eb9f-4c0e-bb60-59474603862a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:40.945202 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7e7c2-eb9f-4c0e-bb60-59474603862a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:40.945469 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7e7c2-eb9f-4c0e-bb60-59474603862a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:40.957503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7e7c2-eb9f-4c0e-bb60-59474603862a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:40.959145 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7e7c2-eb9f-4c0e-bb60-59474603862a None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:40.975878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-48c7e7c2-eb9f-4c0e-bb60-59474603862a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:40.975878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:40.975878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:40.975878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:40.975878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:40.975878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:40.975878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:40.975878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:40.975878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:40.975878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:40.975878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:40.975878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:40.975878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:40.975878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:40.975878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:40.977168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:40.977168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:40.977168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:40.977168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:40.977168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:40.977168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:40.977168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:40.977168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:40.977168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:40.977168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:40.977168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:40.977168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:40.977168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:40.977168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:40.977168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:40.977168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:40.977168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:40.978923 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:40.978923 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:40.978923 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:40.978923 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:40.978923 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 353/713] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:40.978923 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7e7c2-eb9f-4c0e-bb60-59474603862a None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:40.983218 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1503d-f13f-4b91-a03f-84585def5477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:40.983871 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1503d-f13f-4b91-a03f-84585def5477 None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:40.984146 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1503d-f13f-4b91-a03f-84585def5477 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:40.984415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1503d-f13f-4b91-a03f-84585def5477 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:40.985234 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1503d-f13f-4b91-a03f-84585def5477 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:40.986524 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1503d-f13f-4b91-a03f-84585def5477 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:41.007003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 361/714] 10.222.0.93 () {60 vars in 1141 bytes} [Mon Aug 1 14:58:40 2022] POST /identity/v3/services => generated 251 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:58:41.007870 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1503d-f13f-4b91-a03f-84585def5477 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:41.019687 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:41.118786 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7e7c2-eb9f-4c0e-bb60-59474603862a None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:41.170719 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db9b5f-10af-4d44-83dc-3fbdd69669a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:41.171486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db9b5f-10af-4d44-83dc-3fbdd69669a0 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:41.172000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db9b5f-10af-4d44-83dc-3fbdd69669a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:41.172369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db9b5f-10af-4d44-83dc-3fbdd69669a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:41.190783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db9b5f-10af-4d44-83dc-3fbdd69669a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:41.194488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db9b5f-10af-4d44-83dc-3fbdd69669a0 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:41.210014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1503d-f13f-4b91-a03f-84585def5477 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:41.228476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-c5db9b5f-10af-4d44-83dc-3fbdd69669a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:41.228476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:41.228476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:41.228476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:41.228476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:41.228476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:41.228476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:41.228476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:41.228476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:41.228476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:41.228476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:41.228476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:41.228476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:41.228476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:41.228476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:41.229678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:41.229678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:41.229678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:41.229678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:41.229678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:41.229678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:41.229678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:41.229678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:41.229678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:41.229678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:41.229678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:41.229678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:41.229678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:41.229678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:41.229678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:41.229678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:41.229678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:41.230633 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:41.230633 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:41.230633 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:41.230633 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:41.230633 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 354/715] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:41.234054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db9b5f-10af-4d44-83dc-3fbdd69669a0 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:41.246999 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:41.273174 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4cbfc-87d3-440d-8468-c73d5717a801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:41.273952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4cbfc-87d3-440d-8468-c73d5717a801 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:41.274239 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4cbfc-87d3-440d-8468-c73d5717a801 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:41.274509 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4cbfc-87d3-440d-8468-c73d5717a801 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:41.275201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4cbfc-87d3-440d-8468-c73d5717a801 None admin] RBAC: Adding query filter params (type=network) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:41.275504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4cbfc-87d3-440d-8468-c73d5717a801 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:41.277151 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4cbfc-87d3-440d-8468-c73d5717a801 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:41.283781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 362/716] 10.222.0.93 () {56 vars in 1112 bytes} [Mon Aug 1 14:58:41 2022] GET /identity/v3/services?type=network => generated 124 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:41.284828 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4cbfc-87d3-440d-8468-c73d5717a801 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:41.375496 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:41.416424 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db9b5f-10af-4d44-83dc-3fbdd69669a0 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:41.440305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4cbfc-87d3-440d-8468-c73d5717a801 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:41.463716 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b410a8f-1dcd-4b34-87cc-c6f0f00d6fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:41.465104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b410a8f-1dcd-4b34-87cc-c6f0f00d6fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:41.465881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b410a8f-1dcd-4b34-87cc-c6f0f00d6fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:41.466725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b410a8f-1dcd-4b34-87cc-c6f0f00d6fd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:41.467986 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b410a8f-1dcd-4b34-87cc-c6f0f00d6fd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:41.468653 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b410a8f-1dcd-4b34-87cc-c6f0f00d6fd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:41.470529 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b410a8f-1dcd-4b34-87cc-c6f0f00d6fd2 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:41.487864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e1864-08d8-468d-83e1-7a5af97b522e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:41.488570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e1864-08d8-468d-83e1-7a5af97b522e None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:41.488818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e1864-08d8-468d-83e1-7a5af97b522e None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:41.489643 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e1864-08d8-468d-83e1-7a5af97b522e None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:41.490360 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e1864-08d8-468d-83e1-7a5af97b522e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:41.490668 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e1864-08d8-468d-83e1-7a5af97b522e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:41.492207 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e1864-08d8-468d-83e1-7a5af97b522e None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:41.498559 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 355/717] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:41.507173 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b410a8f-1dcd-4b34-87cc-c6f0f00d6fd2 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:41.522875 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 363/718] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:41.532342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e1864-08d8-468d-83e1-7a5af97b522e None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:41.656664 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:41.664578 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b410a8f-1dcd-4b34-87cc-c6f0f00d6fd2 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:41.714598 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf65eac-99fe-41ff-928a-e06b72b713a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:41.715742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf65eac-99fe-41ff-928a-e06b72b713a2 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:41.717165 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf65eac-99fe-41ff-928a-e06b72b713a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:41.717753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf65eac-99fe-41ff-928a-e06b72b713a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:41.728437 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf65eac-99fe-41ff-928a-e06b72b713a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:41.731893 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf65eac-99fe-41ff-928a-e06b72b713a2 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:41.734828 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:41.752513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-2bf65eac-99fe-41ff-928a-e06b72b713a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:41.752513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:41.752513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:41.752513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:41.752513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:41.752513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:41.752513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:41.752513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:41.752513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:41.752513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:41.752513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:41.752513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:41.752513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:41.752513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:41.752513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:41.753729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:41.753729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:41.753729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:41.753729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:41.753729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:41.753729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:41.753729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:41.753729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:41.753729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:41.753729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:41.753729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:41.753729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:41.753729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:41.753729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:41.753729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:41.753729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:41.753729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:41.755406 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:41.755406 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:41.755406 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:41.755406 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:41.755406 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:41.755406 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:41.755406 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:41.755406 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:41.758454 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 356/719] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:41.767451 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf65eac-99fe-41ff-928a-e06b72b713a2 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:41.826199 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e1864-08d8-468d-83e1-7a5af97b522e None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:41.904889 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf65eac-99fe-41ff-928a-e06b72b713a2 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:41.925598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abfa0a-4d0d-4dac-95fa-723729d07eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:41.925957 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abfa0a-4d0d-4dac-95fa-723729d07eab None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:41.926189 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abfa0a-4d0d-4dac-95fa-723729d07eab None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:41.926486 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abfa0a-4d0d-4dac-95fa-723729d07eab None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:41.927753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abfa0a-4d0d-4dac-95fa-723729d07eab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:41.929324 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abfa0a-4d0d-4dac-95fa-723729d07eab None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:41.954199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f18ac5e-2f34-40fa-987e-038cff3b123f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:41.955423 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18ac5e-2f34-40fa-987e-038cff3b123f None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:41.955951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18ac5e-2f34-40fa-987e-038cff3b123f None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:41.956517 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18ac5e-2f34-40fa-987e-038cff3b123f None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:41.957494 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f18ac5e-2f34-40fa-987e-038cff3b123f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:41.958089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f18ac5e-2f34-40fa-987e-038cff3b123f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:41.960660 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f18ac5e-2f34-40fa-987e-038cff3b123f None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:41.978434 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 357/720] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:41.983990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 364/721] 10.222.0.93 () {60 vars in 1132 bytes} [Mon Aug 1 14:58:41 2022] POST /identity/v3/users => generated 254 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:58:41.989067 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f18ac5e-2f34-40fa-987e-038cff3b123f None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:42.100172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:42.175580 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f18ac5e-2f34-40fa-987e-038cff3b123f None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:42.240487 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe60d81-4d37-4eae-91c7-2c28cc4c8af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:42.241217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe60d81-4d37-4eae-91c7-2c28cc4c8af3 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:42.241599 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe60d81-4d37-4eae-91c7-2c28cc4c8af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:42.241949 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe60d81-4d37-4eae-91c7-2c28cc4c8af3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:42.243149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe60d81-4d37-4eae-91c7-2c28cc4c8af3 None admin] RBAC: Adding query filter params (role.id=5456887a306649bf80a9ea7bd4282e97, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=3d9be74621d14c94b95ae94947e5abbd) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:42.243609 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe60d81-4d37-4eae-91c7-2c28cc4c8af3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:42.246303 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe60d81-4d37-4eae-91c7-2c28cc4c8af3 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:42.254854 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 358/722] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:58:41 2022] GET /identity/v3/role_assignments?user.id=3d9be74621d14c94b95ae94947e5abbd&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:42.521976 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171221c4-bce8-42d5-8d35-e8b6a842fd6d None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:42.522525 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171221c4-bce8-42d5-8d35-e8b6a842fd6d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:42.522525 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171221c4-bce8-42d5-8d35-e8b6a842fd6d None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:42.524236 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 365/723] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:42.529719 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfef5a-538e-41f4-a673-aee2461379e2 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:42.530000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfef5a-538e-41f4-a673-aee2461379e2 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:42.530273 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfef5a-538e-41f4-a673-aee2461379e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:42.553384 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-c4cfef5a-538e-41f4-a673-aee2461379e2 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:42.554610 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c4cfef5a-538e-41f4-a673-aee2461379e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:42.636230 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 359/724] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:42 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:42.660191 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171221c4-bce8-42d5-8d35-e8b6a842fd6d None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:42.847772 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:42.859163 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171221c4-bce8-42d5-8d35-e8b6a842fd6d None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:42.907541 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02900431-605b-467f-83f9-53c239d469ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:42.908185 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02900431-605b-467f-83f9-53c239d469ec None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:42.908479 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02900431-605b-467f-83f9-53c239d469ec None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:42.908754 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02900431-605b-467f-83f9-53c239d469ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:42.925497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02900431-605b-467f-83f9-53c239d469ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:42.931520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-02900431-605b-467f-83f9-53c239d469ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:58:42.931520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:42.931520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:42.931520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:42.931520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:42.931520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:42.931520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:42.931520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:42.931520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:42.931520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:42.931520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:42.931520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:42.931520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:42.931520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:42.931520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:42.931520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:42.932606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:42.932606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:42.932606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:42.932606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:42.932606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:42.932606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:42.932606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:42.932606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:42.932606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:42.932606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:42.932606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:42.932606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:42.932606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:42.932606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:42.932606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:42.932606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:42.933545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:42.933545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:42.933545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:42.933545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:42.933545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:42.933545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:42.933545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:42.933545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:42.933545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:42.933545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:42.933545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:58:42.933545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:42.933545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 366/725] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:58:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:42.938511 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cfef5a-538e-41f4-a673-aee2461379e2 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:43.014910 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:43.069415 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cfef5a-538e-41f4-a673-aee2461379e2 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:43.112076 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73754a85-c8b6-4644-81af-1c3df2dede77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:43.112837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73754a85-c8b6-4644-81af-1c3df2dede77 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:43.113013 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73754a85-c8b6-4644-81af-1c3df2dede77 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:43.113175 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73754a85-c8b6-4644-81af-1c3df2dede77 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:43.113870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73754a85-c8b6-4644-81af-1c3df2dede77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:43.114216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73754a85-c8b6-4644-81af-1c3df2dede77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:43.115587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73754a85-c8b6-4644-81af-1c3df2dede77 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:43.125032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 360/726] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:58:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:43.130517 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02900431-605b-467f-83f9-53c239d469ec None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:43.273541 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02900431-605b-467f-83f9-53c239d469ec None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:43.327152 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57cd3a-aa8f-4d09-9802-6f998edd2d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:43.327803 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57cd3a-aa8f-4d09-9802-6f998edd2d96 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:43.328079 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57cd3a-aa8f-4d09-9802-6f998edd2d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:43.328349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57cd3a-aa8f-4d09-9802-6f998edd2d96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:43.340488 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57cd3a-aa8f-4d09-9802-6f998edd2d96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:43.341969 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57cd3a-aa8f-4d09-9802-6f998edd2d96 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:43.364958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-da57cd3a-aa8f-4d09-9802-6f998edd2d96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:43.364958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:43.364958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:43.364958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:43.364958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:43.364958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:43.364958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:43.364958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:43.364958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:43.364958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:43.364958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:43.364958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:43.364958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:43.364958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:43.364958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:43.367073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:43.367073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:43.367073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:43.367073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:43.367073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:43.367073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:43.367073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:43.367073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:43.367073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:43.367073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:43.367073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:43.367073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:43.367073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:43.367073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:43.367073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:43.367073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:43.367073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:43.369436 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:43.369436 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:43.369436 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:43.369436 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:43.369436 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 367/727] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:43.377629 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73754a85-c8b6-4644-81af-1c3df2dede77 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:43.536940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73754a85-c8b6-4644-81af-1c3df2dede77 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:43.561647 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:43.590157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e885486-9fcd-4eca-9d5e-73c9e26f19c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:43.590783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e885486-9fcd-4eca-9d5e-73c9e26f19c0 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:43.591062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e885486-9fcd-4eca-9d5e-73c9e26f19c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:43.591332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e885486-9fcd-4eca-9d5e-73c9e26f19c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:43.592073 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e885486-9fcd-4eca-9d5e-73c9e26f19c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:43.592380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e885486-9fcd-4eca-9d5e-73c9e26f19c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:43.593779 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e885486-9fcd-4eca-9d5e-73c9e26f19c0 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:43.615558 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 361/728] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:43.621031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57cd3a-aa8f-4d09-9802-6f998edd2d96 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:43.650046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:43.689637 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c6fa7-a218-4257-87e3-cc9e417a79a7 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:43.689973 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c6fa7-a218-4257-87e3-cc9e417a79a7 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:43.690199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c6fa7-a218-4257-87e3-cc9e417a79a7 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:43.691796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 362/729] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:43.697058 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0a2f4-8f03-422c-9a4b-dbaa4988985c None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:43.697341 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0a2f4-8f03-422c-9a4b-dbaa4988985c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:43.697616 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0a2f4-8f03-422c-9a4b-dbaa4988985c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:43.717981 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-f5d0a2f4-8f03-422c-9a4b-dbaa4988985c None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:43.719477 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-f5d0a2f4-8f03-422c-9a4b-dbaa4988985c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:43.793064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57cd3a-aa8f-4d09-9802-6f998edd2d96 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:43.857129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b8755-ffd5-4a87-897b-e5647a3d9886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:43.857883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b8755-ffd5-4a87-897b-e5647a3d9886 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:43.858260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b8755-ffd5-4a87-897b-e5647a3d9886 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:43.858542 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b8755-ffd5-4a87-897b-e5647a3d9886 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:43.867948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b8755-ffd5-4a87-897b-e5647a3d9886 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:43.869589 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b8755-ffd5-4a87-897b-e5647a3d9886 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:43.882191 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 363/730] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:43 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:58:43.893864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-099b8755-ffd5-4a87-897b-e5647a3d9886 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:58:43.893864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:43.893864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:43.893864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:43.893864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:43.893864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:43.893864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:43.893864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:43.893864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:43.893864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:43.893864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:43.893864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:43.893864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:43.893864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:43.893864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:43.893864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:43.894891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:43.894891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:43.894891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:43.894891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:43.894891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:43.894891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:43.894891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:43.894891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:43.894891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:43.894891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:43.894891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:43.894891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:43.894891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:43.894891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:43.894891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:43.894891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:43.894891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:43.894891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:43.896598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:43.896598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:43.896598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:43.896598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:43.896598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:43.896598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:43.896598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:43.896598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:43.896598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:58:43.896598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:43.896598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 368/731] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:58:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:43.901229 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0a2f4-8f03-422c-9a4b-dbaa4988985c None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:43.912705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b8755-ffd5-4a87-897b-e5647a3d9886 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:44.014237 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:44.047788 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b8755-ffd5-4a87-897b-e5647a3d9886 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:44.072910 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0a2f4-8f03-422c-9a4b-dbaa4988985c None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:44.076418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:44.086488 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7f167-d9da-49c8-ac38-2e00f8284231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:44.087224 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7f167-d9da-49c8-ac38-2e00f8284231 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:44.087500 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7f167-d9da-49c8-ac38-2e00f8284231 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:44.087769 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7f167-d9da-49c8-ac38-2e00f8284231 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:44.088385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7f167-d9da-49c8-ac38-2e00f8284231 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:44.089773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7f167-d9da-49c8-ac38-2e00f8284231 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:44.098971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-b7f7f167-d9da-49c8-ac38-2e00f8284231 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 14:58:44.098971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:44.098971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:58:44.098971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:58:44.098971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:58:44.098971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:58:44.098971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:58:44.098971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:58:44.098971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 14:58:44.098971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:44.098971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:58:44.098971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:44.098971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:44.098971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:44.098971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:44.098971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:44.100021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:44.100021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:44.100021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:44.100021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:44.100021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:44.100021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:44.100021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:44.100021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:58:44.100021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:58:44.100021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:58:44.100021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:58:44.100021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:44.100021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:44.100021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:44.100021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:44.100021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:44.100939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:44.100939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:44.100939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:44.100939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:44.100939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:44.100939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:44.100939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:44.100939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:44.100939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:44.100939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:44.100939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:44.100939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:58:44.100939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:58:44.100939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 14:58:44.100939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:44.100939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 369/732] 10.222.0.93 () {56 vars in 1129 bytes} [Mon Aug 1 14:58:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:58:44.102365 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a4994-d77b-4d95-98e7-84b5ae3ac019 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:44.102365 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a4994-d77b-4d95-98e7-84b5ae3ac019 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:44.102365 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a4994-d77b-4d95-98e7-84b5ae3ac019 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:44.107597 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 370/733] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:44.108568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a4994-d77b-4d95-98e7-84b5ae3ac019 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:44.117170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e138082-157c-4d3c-8e3e-11689b76472c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:44.117790 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e138082-157c-4d3c-8e3e-11689b76472c None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:44.118125 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e138082-157c-4d3c-8e3e-11689b76472c None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:44.118343 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e138082-157c-4d3c-8e3e-11689b76472c None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:44.119731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e138082-157c-4d3c-8e3e-11689b76472c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:44.120042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e138082-157c-4d3c-8e3e-11689b76472c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:44.122285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e138082-157c-4d3c-8e3e-11689b76472c None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:44.147722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 364/734] 10.222.0.93 () {56 vars in 1135 bytes} [Mon Aug 1 14:58:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:44.149088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f782c8-26d0-4916-8c43-90bc7f96b756 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:44.149379 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f782c8-26d0-4916-8c43-90bc7f96b756 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:44.149652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f782c8-26d0-4916-8c43-90bc7f96b756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:44.176999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-61f782c8-26d0-4916-8c43-90bc7f96b756 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:44.178489 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-61f782c8-26d0-4916-8c43-90bc7f96b756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:44.239414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a4994-d77b-4d95-98e7-84b5ae3ac019 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:44.264613 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 365/735] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:44 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:58:44.265570 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f782c8-26d0-4916-8c43-90bc7f96b756 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:44.283849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf1b5e-d75a-4b94-9ad2-a7fc6018444b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:44.284489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf1b5e-d75a-4b94-9ad2-a7fc6018444b None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:44.284771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf1b5e-d75a-4b94-9ad2-a7fc6018444b None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:44.285046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf1b5e-d75a-4b94-9ad2-a7fc6018444b None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:44.285721 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf1b5e-d75a-4b94-9ad2-a7fc6018444b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:44.286543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf1b5e-d75a-4b94-9ad2-a7fc6018444b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:44.289106 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf1b5e-d75a-4b94-9ad2-a7fc6018444b None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:44.291529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:44.295772 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 371/736] 10.222.0.93 () {56 vars in 1124 bytes} [Mon Aug 1 14:58:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:44.301484 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf1b5e-d75a-4b94-9ad2-a7fc6018444b None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:44.388816 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:44.400710 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f782c8-26d0-4916-8c43-90bc7f96b756 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:44.440445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a6806-7b79-49ce-9df7-4578731fe39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:44.441140 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a6806-7b79-49ce-9df7-4578731fe39a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:44.441415 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a6806-7b79-49ce-9df7-4578731fe39a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:44.442749 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a6806-7b79-49ce-9df7-4578731fe39a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:44.446391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf1b5e-d75a-4b94-9ad2-a7fc6018444b None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:44.523346 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a6806-7b79-49ce-9df7-4578731fe39a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:44.525230 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a6806-7b79-49ce-9df7-4578731fe39a None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:44.540096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db28a0-a5e8-4392-bdb3-e2ab816af047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:44.541203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db28a0-a5e8-4392-bdb3-e2ab816af047 None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:44.541597 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db28a0-a5e8-4392-bdb3-e2ab816af047 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:44.542925 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db28a0-a5e8-4392-bdb3-e2ab816af047 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:44.543630 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db28a0-a5e8-4392-bdb3-e2ab816af047 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:44.545922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db28a0-a5e8-4392-bdb3-e2ab816af047 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:44.572518 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 372/737] 10.222.0.93 () {60 vars in 1141 bytes} [Mon Aug 1 14:58:44 2022] POST /identity/v3/services => generated 240 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:58:44.572518 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db28a0-a5e8-4392-bdb3-e2ab816af047 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:44.677557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:44.689226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 366/738] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:44 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:58:44.690656 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:44.696020 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a6806-7b79-49ce-9df7-4578731fe39a None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:44.811177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a6806-7b79-49ce-9df7-4578731fe39a None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:44.818693 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db28a0-a5e8-4392-bdb3-e2ab816af047 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:44.855567 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa9450-325f-48df-83e5-c9847519207c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:44.856090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa9450-325f-48df-83e5-c9847519207c None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:44.856369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa9450-325f-48df-83e5-c9847519207c None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:44.856651 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa9450-325f-48df-83e5-c9847519207c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:44.869975 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0687ee4-9156-4463-89eb-2160b83c8015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:44.870360 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa9450-325f-48df-83e5-c9847519207c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:44.870889 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0687ee4-9156-4463-89eb-2160b83c8015 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:44.871176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0687ee4-9156-4463-89eb-2160b83c8015 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:44.871450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0687ee4-9156-4463-89eb-2160b83c8015 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:44.872149 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0687ee4-9156-4463-89eb-2160b83c8015 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:44.872492 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0687ee4-9156-4463-89eb-2160b83c8015 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:44.873493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa9450-325f-48df-83e5-c9847519207c None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:44.873930 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0687ee4-9156-4463-89eb-2160b83c8015 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:44.881577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 373/739] 10.222.0.93 () {56 vars in 1124 bytes} [Mon Aug 1 14:58:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:44.883058 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefc01f-4060-40c3-a34e-53ae99e9013e None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:44.883340 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefc01f-4060-40c3-a34e-53ae99e9013e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:44.883617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefc01f-4060-40c3-a34e-53ae99e9013e None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:44.885249 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 374/740] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:44.891699 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fefc01f-4060-40c3-a34e-53ae99e9013e None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:44.898027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-57aa9450-325f-48df-83e5-c9847519207c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:44.898027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:44.898027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:44.898027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:44.898027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:44.898027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:44.898027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:44.898027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:44.898027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:44.898027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:44.898027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:44.898027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:44.898027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:44.898027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:44.898027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:44.898974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:44.898974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:44.898974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:44.898974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:44.898974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:44.898974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:44.898974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:44.898974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:44.898974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:44.898974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:44.898974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:44.898974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:44.898974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:44.898974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:44.898974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:44.898974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:44.898974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:44.899891 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:44.899891 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:44.899891 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:44.899891 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:44.900605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 367/741] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:44.903188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76b8ab-8cb6-46bc-85f4-aef53bf0a632 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:44.903561 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76b8ab-8cb6-46bc-85f4-aef53bf0a632 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:44.903847 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76b8ab-8cb6-46bc-85f4-aef53bf0a632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:44.929775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-8f76b8ab-8cb6-46bc-85f4-aef53bf0a632 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:44.931214 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-8f76b8ab-8cb6-46bc-85f4-aef53bf0a632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:44.978771 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:45.012510 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 368/742] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:58:45.012510 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76b8ab-8cb6-46bc-85f4-aef53bf0a632 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:45.034148 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fefc01f-4060-40c3-a34e-53ae99e9013e None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:45.062986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:45.082768 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67b46f-fb04-46ac-b780-080cfbe6ddd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:45.083399 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67b46f-fb04-46ac-b780-080cfbe6ddd5 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:45.083815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67b46f-fb04-46ac-b780-080cfbe6ddd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:45.084110 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67b46f-fb04-46ac-b780-080cfbe6ddd5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:45.085510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67b46f-fb04-46ac-b780-080cfbe6ddd5 None admin] RBAC: Adding query filter params (interface=public, service_id=d7e36cbe6f42421f9b2a69eaab3a916c, region_id=RegionOne) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:45.085733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67b46f-fb04-46ac-b780-080cfbe6ddd5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:45.087439 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67b46f-fb04-46ac-b780-080cfbe6ddd5 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:45.096697 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 375/743] 10.222.0.93 () {56 vars in 1251 bytes} [Mon Aug 1 14:58:44 2022] GET /identity/v3/endpoints?service_id=d7e36cbe6f42421f9b2a69eaab3a916c&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:45.098941 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b10b8a-d3d9-49e4-8d14-06f9b4e2fab7 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:45.099468 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b10b8a-d3d9-49e4-8d14-06f9b4e2fab7 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:45.099929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b10b8a-d3d9-49e4-8d14-06f9b4e2fab7 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:45.101706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 376/744] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:45.102680 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b10b8a-d3d9-49e4-8d14-06f9b4e2fab7 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:45.132374 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76b8ab-8cb6-46bc-85f4-aef53bf0a632 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:45.179053 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfb4e6-86b1-4046-aa27-d684a0562c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:45.179678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfb4e6-86b1-4046-aa27-d684a0562c5f None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:45.180086 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfb4e6-86b1-4046-aa27-d684a0562c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:45.180304 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfb4e6-86b1-4046-aa27-d684a0562c5f None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:45.181177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cfb4e6-86b1-4046-aa27-d684a0562c5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:45.181475 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cfb4e6-86b1-4046-aa27-d684a0562c5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:45.183004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cfb4e6-86b1-4046-aa27-d684a0562c5f None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:45.200588 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 369/745] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:45.201478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfb4e6-86b1-4046-aa27-d684a0562c5f None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:45.222775 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b10b8a-d3d9-49e4-8d14-06f9b4e2fab7 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:45.248297 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:45.265775 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fa122-6636-4604-9eb2-10ff89ff13a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:45.266442 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fa122-6636-4604-9eb2-10ff89ff13a9 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:45.266730 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fa122-6636-4604-9eb2-10ff89ff13a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:45.267001 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fa122-6636-4604-9eb2-10ff89ff13a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:45.279305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316fa122-6636-4604-9eb2-10ff89ff13a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:45.284227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-316fa122-6636-4604-9eb2-10ff89ff13a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:58:45.284227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:45.284227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:45.284227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:45.284227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:45.284227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:45.284227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:45.284227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:45.284227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:45.284227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:45.284227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:45.284227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:45.284227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:45.284227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:45.284227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:45.284227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:45.285252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:45.285252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:45.285252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:45.285252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:45.285252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:45.285252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:45.285252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:45.285252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:45.285252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:45.285252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:45.285252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:45.285252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:45.285252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:45.285252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:45.285252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:45.285252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:45.286715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:45.286715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:45.286715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:45.286715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:45.286715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:45.286715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:45.286715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:45.286715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:45.286715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:45.286715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:45.286715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:58:45.286715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:45.286715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 377/746] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:58:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:45.287443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a7566-abc3-47f2-95a3-c4037b496a7e None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:45.287878 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a7566-abc3-47f2-95a3-c4037b496a7e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:45.287878 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a7566-abc3-47f2-95a3-c4037b496a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:45.306735 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-b20a7566-abc3-47f2-95a3-c4037b496a7e None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:45.307929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b20a7566-abc3-47f2-95a3-c4037b496a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:45.335258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfb4e6-86b1-4046-aa27-d684a0562c5f None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:45.354193 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:45.380609 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89445567-b746-4478-97e9-793b657a6170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:45.381984 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89445567-b746-4478-97e9-793b657a6170 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:45.382325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89445567-b746-4478-97e9-793b657a6170 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:45.383063 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89445567-b746-4478-97e9-793b657a6170 None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:45.383839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89445567-b746-4478-97e9-793b657a6170 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:45.386830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89445567-b746-4478-97e9-793b657a6170 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:45.391151 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 378/747] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:58:45.392231 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a7566-abc3-47f2-95a3-c4037b496a7e None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:45.398680 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 370/748] 10.222.0.93 () {56 vars in 1087 bytes} [Mon Aug 1 14:58:45 2022] GET /identity/v3/services => generated 772 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:45.399540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89445567-b746-4478-97e9-793b657a6170 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:45.559694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a7566-abc3-47f2-95a3-c4037b496a7e None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:45.574561 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:45.581230 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89445567-b746-4478-97e9-793b657a6170 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:45.620878 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c36257-47c3-435e-af0a-8e1bd1d0300b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:45.621987 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c36257-47c3-435e-af0a-8e1bd1d0300b None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:45.622213 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c36257-47c3-435e-af0a-8e1bd1d0300b None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:45.622489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c36257-47c3-435e-af0a-8e1bd1d0300b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:45.637118 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c36257-47c3-435e-af0a-8e1bd1d0300b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:45.638859 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c36257-47c3-435e-af0a-8e1bd1d0300b None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:45.648094 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f0c32-7d7e-4583-9500-eadacd69c379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:45.649207 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f0c32-7d7e-4583-9500-eadacd69c379 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:45.649908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f0c32-7d7e-4583-9500-eadacd69c379 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:45.650473 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f0c32-7d7e-4583-9500-eadacd69c379 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:45.651939 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f0c32-7d7e-4583-9500-eadacd69c379 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:45.652574 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f0c32-7d7e-4583-9500-eadacd69c379 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:45.653891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-17c36257-47c3-435e-af0a-8e1bd1d0300b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:45.653891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:45.653891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:45.653891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:45.653891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:45.653891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:45.653891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:45.653891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:45.653891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:45.653891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:45.653891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:45.653891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:45.653891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:45.653891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:45.653891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:45.655134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:45.655134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:45.655134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:45.655134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:45.655134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:45.655134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:45.655134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:45.655134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:45.655134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:45.655134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:45.655134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:45.655134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:45.655134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:45.655134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:45.655134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:45.655134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:45.655134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:45.656549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:45.656549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:45.656549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:45.656549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:45.656549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:45.656549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:45.656549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:45.656549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:45.656549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 379/749] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:45.657252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c36257-47c3-435e-af0a-8e1bd1d0300b None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:45.658541 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f0c32-7d7e-4583-9500-eadacd69c379 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:45.672862 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 371/750] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:58:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:45.674163 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f0c32-7d7e-4583-9500-eadacd69c379 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:45.711368 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:45.808310 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c36257-47c3-435e-af0a-8e1bd1d0300b None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:45.832064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f0c32-7d7e-4583-9500-eadacd69c379 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:45.850987 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675457d-b03a-4f90-892e-9e34f6e841b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:45.851982 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675457d-b03a-4f90-892e-9e34f6e841b6 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:45.852439 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675457d-b03a-4f90-892e-9e34f6e841b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:45.852916 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675457d-b03a-4f90-892e-9e34f6e841b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:45.864718 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c675457d-b03a-4f90-892e-9e34f6e841b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:45.871965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-c675457d-b03a-4f90-892e-9e34f6e841b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:45.871965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:45.871965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:45.871965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:45.871965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:45.871965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:45.871965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:45.871965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:45.871965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:45.871965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:45.871965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:45.871965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:45.871965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:45.871965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:45.871965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:45.871965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:45.873160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:45.873160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:45.873160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:45.873160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:45.873160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:45.873160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:45.873160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:45.873160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:45.873160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:45.873160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:45.873160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:45.873160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:45.873160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:45.873160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:45.873160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:45.873160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:45.874150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:45.874150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:45.874150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:45.874150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:45.874150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:45.874150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:45.874150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:45.874150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:45.874150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:45.874150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:45.874150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:45.874150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:45.877286 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 380/751] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:45.878445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675457d-b03a-4f90-892e-9e34f6e841b6 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:45.880591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85902836-3fad-4d90-a784-49960a38edcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:45.881450 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85902836-3fad-4d90-a784-49960a38edcc None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:45.881985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85902836-3fad-4d90-a784-49960a38edcc None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:45.882417 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85902836-3fad-4d90-a784-49960a38edcc None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:45.883711 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85902836-3fad-4d90-a784-49960a38edcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:45.885207 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85902836-3fad-4d90-a784-49960a38edcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:45.886852 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85902836-3fad-4d90-a784-49960a38edcc None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:45.906499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 372/752] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:45.907905 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85902836-3fad-4d90-a784-49960a38edcc None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:45.937794 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:45.995658 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:46.003593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675457d-b03a-4f90-892e-9e34f6e841b6 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:46.039519 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85902836-3fad-4d90-a784-49960a38edcc None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:46.041083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7153fa1-6fe2-429d-a914-a3f0b901857c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:46.041825 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7153fa1-6fe2-429d-a914-a3f0b901857c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:46.042201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7153fa1-6fe2-429d-a914-a3f0b901857c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:46.043246 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7153fa1-6fe2-429d-a914-a3f0b901857c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:46.054614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7153fa1-6fe2-429d-a914-a3f0b901857c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:46.056243 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7153fa1-6fe2-429d-a914-a3f0b901857c None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:46.073945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-a7153fa1-6fe2-429d-a914-a3f0b901857c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:46.073945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:46.073945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:46.073945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:46.073945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:46.073945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:46.073945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:46.073945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:46.073945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:46.073945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:46.073945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:46.073945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:46.073945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:46.073945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:46.073945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:46.075073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:46.075073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:46.075073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:46.075073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:46.075073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:46.075073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:46.075073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:46.075073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:46.075073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:46.075073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:46.075073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:46.075073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:46.075073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:46.075073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:46.075073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:46.075073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:46.075073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:46.076027 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:46.076027 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:46.076027 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:46.076027 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:46.076651 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 381/753] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:46.077830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7153fa1-6fe2-429d-a914-a3f0b901857c None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:46.083641 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92637d80-7e10-4ad0-b343-2f180b810b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:46.084677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92637d80-7e10-4ad0-b343-2f180b810b51 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:46.085134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92637d80-7e10-4ad0-b343-2f180b810b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:46.085614 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92637d80-7e10-4ad0-b343-2f180b810b51 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:46.086528 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92637d80-7e10-4ad0-b343-2f180b810b51 None admin] RBAC: Adding query filter params (name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:46.087048 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92637d80-7e10-4ad0-b343-2f180b810b51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:46.089070 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92637d80-7e10-4ad0-b343-2f180b810b51 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:46.100945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 373/754] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 14:58:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:46.102122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92637d80-7e10-4ad0-b343-2f180b810b51 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:46.189355 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7153fa1-6fe2-429d-a914-a3f0b901857c None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:46.196845 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:46.213138 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92637d80-7e10-4ad0-b343-2f180b810b51 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:46.225587 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7efe0-d99c-4de1-9dd2-2ede8fe43da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:46.226441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7efe0-d99c-4de1-9dd2-2ede8fe43da7 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:46.226604 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7efe0-d99c-4de1-9dd2-2ede8fe43da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:46.226967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7efe0-d99c-4de1-9dd2-2ede8fe43da7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:46.227820 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7efe0-d99c-4de1-9dd2-2ede8fe43da7 None admin] RBAC: Adding query filter params (role.id=5456887a306649bf80a9ea7bd4282e97, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=4df37b02152c48509243b73ebbf49d8d) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:46.228194 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7efe0-d99c-4de1-9dd2-2ede8fe43da7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:46.229829 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7efe0-d99c-4de1-9dd2-2ede8fe43da7 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:46.236224 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 382/755] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:58:46 2022] GET /identity/v3/role_assignments?user.id=4df37b02152c48509243b73ebbf49d8d&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:46.237236 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7efe0-d99c-4de1-9dd2-2ede8fe43da7 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:46.251900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:46.257700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638afc9c-08ed-4623-9b1a-d1dd2e60d700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:46.258697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638afc9c-08ed-4623-9b1a-d1dd2e60d700 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:46.259157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638afc9c-08ed-4623-9b1a-d1dd2e60d700 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:46.259603 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638afc9c-08ed-4623-9b1a-d1dd2e60d700 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:46.260652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638afc9c-08ed-4623-9b1a-d1dd2e60d700 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:46.261240 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638afc9c-08ed-4623-9b1a-d1dd2e60d700 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:46.262836 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638afc9c-08ed-4623-9b1a-d1dd2e60d700 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:46.277039 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 374/756] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:46.287046 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638afc9c-08ed-4623-9b1a-d1dd2e60d700 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:46.360597 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7efe0-d99c-4de1-9dd2-2ede8fe43da7 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:46.444811 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638afc9c-08ed-4623-9b1a-d1dd2e60d700 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:46.484875 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205f85f-e91a-4b7b-ad6f-55156fd98331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:46.485532 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205f85f-e91a-4b7b-ad6f-55156fd98331 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:46.485826 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205f85f-e91a-4b7b-ad6f-55156fd98331 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:46.486093 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205f85f-e91a-4b7b-ad6f-55156fd98331 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:46.494086 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6f735-7c2d-4fe1-b927-a0bbed4f5337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:46.494925 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6f735-7c2d-4fe1-b927-a0bbed4f5337 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:46.495205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6f735-7c2d-4fe1-b927-a0bbed4f5337 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:46.495725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6f735-7c2d-4fe1-b927-a0bbed4f5337 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:46.508765 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205f85f-e91a-4b7b-ad6f-55156fd98331 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:46.510289 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205f85f-e91a-4b7b-ad6f-55156fd98331 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:46.511603 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6f735-7c2d-4fe1-b927-a0bbed4f5337 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:46.514493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6f735-7c2d-4fe1-b927-a0bbed4f5337 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:46.528392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-7ed6f735-7c2d-4fe1-b927-a0bbed4f5337 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:58:46.528392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:46.528392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:46.528392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:46.528392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:46.528392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:46.528392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:46.528392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:46.528392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:46.528392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:46.528392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:46.528392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:46.528392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:46.528392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:46.528392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:46.528392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:46.529470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:46.529470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:46.529470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:46.529470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:46.529470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:46.529470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:46.529470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:46.529470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:46.529470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:46.529470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:46.529470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:46.529470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:46.529470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:46.529470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:46.529470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:46.529470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:46.529470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:46.529470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:46.530460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:46.530460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:46.530460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:46.530460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:46.530460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:46.530460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:46.530460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:46.530460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:46.530460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:58:46.530460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:46.530460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 375/757] 10.222.0.93 () {56 vars in 1093 bytes} [Mon Aug 1 14:58:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:46.535617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-5205f85f-e91a-4b7b-ad6f-55156fd98331 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:46.535617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:46.535617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:46.535617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:46.535617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:46.535617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:46.535617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:46.535617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:46.535617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:46.535617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:46.535617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:46.535617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:46.535617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:46.535617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:46.535617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:46.537117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:46.537117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:46.537117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:46.537117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:46.537117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:46.537117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:46.537117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:46.537117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:46.537117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:46.537117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:46.537117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:46.537117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:46.537117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:46.537117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:46.537117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:46.537117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:46.537117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:46.538084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:46.538084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:46.538084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:46.538084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:46.539987 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 383/758] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:46.541463 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6f735-7c2d-4fe1-b927-a0bbed4f5337 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:46.546328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205f85f-e91a-4b7b-ad6f-55156fd98331 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:46.546524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:46.684948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:46.694557 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205f85f-e91a-4b7b-ad6f-55156fd98331 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:46.751156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2fc805-e44e-47ea-9e73-6feac17a787c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:46.751921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fc805-e44e-47ea-9e73-6feac17a787c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:46.752116 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fc805-e44e-47ea-9e73-6feac17a787c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:46.752388 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fc805-e44e-47ea-9e73-6feac17a787c None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:46.753130 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2fc805-e44e-47ea-9e73-6feac17a787c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:46.753429 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2fc805-e44e-47ea-9e73-6feac17a787c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:46.754859 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2fc805-e44e-47ea-9e73-6feac17a787c None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:46.758679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6f735-7c2d-4fe1-b927-a0bbed4f5337 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:46.790291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 384/759] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:46.798373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2fc805-e44e-47ea-9e73-6feac17a787c None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:46.816023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec08b1-05ec-4fc0-8129-b935eb0145c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:46.817130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec08b1-05ec-4fc0-8129-b935eb0145c4 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:46.817617 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec08b1-05ec-4fc0-8129-b935eb0145c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:46.818090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec08b1-05ec-4fc0-8129-b935eb0145c4 None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:46.819133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec08b1-05ec-4fc0-8129-b935eb0145c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:46.819658 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec08b1-05ec-4fc0-8129-b935eb0145c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:46.821350 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec08b1-05ec-4fc0-8129-b935eb0145c4 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:46.841346 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 376/760] 10.222.0.93 () {56 vars in 1133 bytes} [Mon Aug 1 14:58:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:46.847246 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec08b1-05ec-4fc0-8129-b935eb0145c4 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:46.932804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:46.983000 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2fc805-e44e-47ea-9e73-6feac17a787c None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:47.015743 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:47.016134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec08b1-05ec-4fc0-8129-b935eb0145c4 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:47.029473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc036a-8cd3-4a18-b570-b40bae058b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:47.030110 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc036a-8cd3-4a18-b570-b40bae058b2b None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:47.030393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc036a-8cd3-4a18-b570-b40bae058b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:47.030671 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc036a-8cd3-4a18-b570-b40bae058b2b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:47.039288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc036a-8cd3-4a18-b570-b40bae058b2b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:47.040932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc036a-8cd3-4a18-b570-b40bae058b2b None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:47.053824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-52dc036a-8cd3-4a18-b570-b40bae058b2b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:58:47.053824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:47.053824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:47.053824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:47.053824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:47.053824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:47.053824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:47.053824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:47.053824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:47.053824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:47.053824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:47.053824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:47.053824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:47.053824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:47.053824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:47.053824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:47.054816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:47.054816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:47.054816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:47.054816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:47.054816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:47.054816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:47.054816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:47.054816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:47.054816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:47.054816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:47.054816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:47.054816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:47.054816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:47.054816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:47.054816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:47.054816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:47.054816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:47.054816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:47.055790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:47.055790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:47.055790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:47.055790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:47.055790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:47.055790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:47.055790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:47.055790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:47.055790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:58:47.055790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:47.055790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 385/761] 10.222.0.93 () {56 vars in 1117 bytes} [Mon Aug 1 14:58:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:47.061014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9737f-12ca-40e9-b56b-48a59df5f238 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:47.061350 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9737f-12ca-40e9-b56b-48a59df5f238 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:47.061636 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9737f-12ca-40e9-b56b-48a59df5f238 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:47.062023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95873a31-078e-4d54-a03f-85be0e6b0de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:47.062891 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95873a31-078e-4d54-a03f-85be0e6b0de7 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:47.063240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 386/762] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:47.063587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95873a31-078e-4d54-a03f-85be0e6b0de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:47.063956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9737f-12ca-40e9-b56b-48a59df5f238 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:47.064406 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95873a31-078e-4d54-a03f-85be0e6b0de7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:47.076116 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95873a31-078e-4d54-a03f-85be0e6b0de7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:47.077980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95873a31-078e-4d54-a03f-85be0e6b0de7 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:47.099290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-95873a31-078e-4d54-a03f-85be0e6b0de7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:47.099290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:47.099290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:47.099290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:47.099290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:47.099290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:47.099290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:47.099290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:47.099290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:47.099290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:47.099290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:47.099290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:47.099290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:47.099290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:47.099290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:47.100254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:47.100254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:47.100254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:47.100254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:47.100254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:47.100254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:47.100254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:47.100254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:47.100254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:47.100254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:47.100254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:47.100254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:47.100254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:47.100254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:47.100254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:47.100254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:47.100254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:47.101203 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:47.101203 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:47.101203 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:47.101203 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:47.107064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 377/763] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:47.107064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09413ac-708b-4840-a78f-c8b1a78d2f33 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:47.107064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09413ac-708b-4840-a78f-c8b1a78d2f33 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:47.107064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09413ac-708b-4840-a78f-c8b1a78d2f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:47.136745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-d09413ac-708b-4840-a78f-c8b1a78d2f33 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:47.139044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d09413ac-708b-4840-a78f-c8b1a78d2f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:47.196956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9737f-12ca-40e9-b56b-48a59df5f238 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:47.236521 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 378/764] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:58:47.237796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09413ac-708b-4840-a78f-c8b1a78d2f33 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:47.250838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f9fac-db5b-42f6-a226-7d394999836f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:47.251792 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f9fac-db5b-42f6-a226-7d394999836f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:47.252761 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f9fac-db5b-42f6-a226-7d394999836f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:47.253236 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f9fac-db5b-42f6-a226-7d394999836f None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:47.254950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f9fac-db5b-42f6-a226-7d394999836f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:47.255429 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f9fac-db5b-42f6-a226-7d394999836f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:47.258011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f9fac-db5b-42f6-a226-7d394999836f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:47.284148 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 387/765] 10.222.0.93 () {56 vars in 1149 bytes} [Mon Aug 1 14:58:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:47.285354 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f9fac-db5b-42f6-a226-7d394999836f None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:47.292246 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:47.344829 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:47.360748 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09413ac-708b-4840-a78f-c8b1a78d2f33 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:47.401892 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b09ab-54c3-4dc8-b073-c225268cb71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:47.402570 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b09ab-54c3-4dc8-b073-c225268cb71d None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:47.402941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b09ab-54c3-4dc8-b073-c225268cb71d None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:47.403318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b09ab-54c3-4dc8-b073-c225268cb71d None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:47.404091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b09ab-54c3-4dc8-b073-c225268cb71d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:47.404509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b09ab-54c3-4dc8-b073-c225268cb71d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:47.405987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b09ab-54c3-4dc8-b073-c225268cb71d None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:47.423746 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 379/766] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:47.423746 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b09ab-54c3-4dc8-b073-c225268cb71d None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:47.430560 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f9fac-db5b-42f6-a226-7d394999836f None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:47.478071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5f31e-dac2-4726-b81b-a0784cfd8e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:47.479284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5f31e-dac2-4726-b81b-a0784cfd8e5f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:47.479731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5f31e-dac2-4726-b81b-a0784cfd8e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:47.480154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5f31e-dac2-4726-b81b-a0784cfd8e5f None admin] PATH_INFO: `/v3/services/network` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:47.481103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5f31e-dac2-4726-b81b-a0784cfd8e5f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:47.482956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5f31e-dac2-4726-b81b-a0784cfd8e5f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:47.497305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-9aa5f31e-dac2-4726-b81b-a0784cfd8e5f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 14:58:47.497305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:47.497305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:58:47.497305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:58:47.497305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:58:47.497305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:58:47.497305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:58:47.497305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:58:47.497305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 14:58:47.497305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:47.497305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:58:47.497305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:47.497305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:47.497305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:47.497305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:47.497305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:47.497305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:47.498493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:47.498493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:47.498493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:47.498493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:47.498493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:47.498493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:47.498493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:58:47.498493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:58:47.498493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:58:47.498493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:58:47.498493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:47.498493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:47.498493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:47.498493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:47.498493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:47.498493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:47.499366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:47.499366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:47.499366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:47.499366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:47.499366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:47.499366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:47.499366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:47.499366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:47.499366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:47.499366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:47.499366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:58:47.499366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:58:47.499366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 14:58:47.499366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:47.502492 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 388/767] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:47 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:47.503526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5f31e-dac2-4726-b81b-a0784cfd8e5f None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:47.551054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b09ab-54c3-4dc8-b073-c225268cb71d None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:47.553624 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:47.601191 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90dd8a0-f670-463b-afef-33d3c3bf815b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:47.601832 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dd8a0-f670-463b-afef-33d3c3bf815b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:47.602113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dd8a0-f670-463b-afef-33d3c3bf815b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:47.602386 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dd8a0-f670-463b-afef-33d3c3bf815b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:47.630868 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5f31e-dac2-4726-b81b-a0784cfd8e5f None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:47.634571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:47.674358 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0a7c7-5ba5-4232-be48-fd528a64b0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:47.675875 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0a7c7-5ba5-4232-be48-fd528a64b0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:47.676991 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0a7c7-5ba5-4232-be48-fd528a64b0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:47.677752 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0a7c7-5ba5-4232-be48-fd528a64b0e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:47.689241 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b0a7c7-5ba5-4232-be48-fd528a64b0e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:47.691759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b0a7c7-5ba5-4232-be48-fd528a64b0e2 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:47.693617 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90dd8a0-f670-463b-afef-33d3c3bf815b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:47.695280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90dd8a0-f670-463b-afef-33d3c3bf815b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:47.708724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-23b0a7c7-5ba5-4232-be48-fd528a64b0e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:47.708724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:47.708724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:47.708724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:47.708724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:47.708724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:47.708724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:47.708724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:47.708724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:47.708724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:47.708724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:47.708724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:47.708724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:47.708724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:47.708724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:47.709698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:47.709698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:47.709698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:47.709698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:47.709698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:47.709698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:47.709698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:47.709698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:47.709698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:47.709698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:47.709698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:47.709698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:47.709698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:47.709698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:47.709698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:47.709698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:47.709698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:47.710639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:47.710639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:47.710639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:47.710639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:47.710639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:47.710639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:47.710639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:47.710639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:47.717238 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 389/768] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:47.717238 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0a7c7-5ba5-4232-be48-fd528a64b0e2 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:47.823637 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 380/769] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:58:47.823637 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90dd8a0-f670-463b-afef-33d3c3bf815b None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:47.827108 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:47.830866 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0a7c7-5ba5-4232-be48-fd528a64b0e2 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:47.867449 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6954c5-689a-4403-b33b-631499720809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:47.868367 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6954c5-689a-4403-b33b-631499720809 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:47.868841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6954c5-689a-4403-b33b-631499720809 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:47.869291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6954c5-689a-4403-b33b-631499720809 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:47.870144 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6954c5-689a-4403-b33b-631499720809 None admin] RBAC: Adding query filter params (name=network) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:47.870714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6954c5-689a-4403-b33b-631499720809 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:47.872106 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6954c5-689a-4403-b33b-631499720809 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:47.877196 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 390/770] 10.222.0.93 () {56 vars in 1112 bytes} [Mon Aug 1 14:58:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:47.883762 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea48ff-a610-4e81-badb-10b78f5f14a2 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:47.884108 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea48ff-a610-4e81-badb-10b78f5f14a2 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:47.884363 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea48ff-a610-4e81-badb-10b78f5f14a2 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:47.888511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 391/771] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:47.888511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea48ff-a610-4e81-badb-10b78f5f14a2 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:47.918727 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:47.964822 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90dd8a0-f670-463b-afef-33d3c3bf815b None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:48.007873 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32654bf-594f-4654-8e95-b56045b67a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:48.008524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32654bf-594f-4654-8e95-b56045b67a45 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:48.008817 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32654bf-594f-4654-8e95-b56045b67a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:48.009093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32654bf-594f-4654-8e95-b56045b67a45 None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:48.009878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32654bf-594f-4654-8e95-b56045b67a45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:48.010188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32654bf-594f-4654-8e95-b56045b67a45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:48.011640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32654bf-594f-4654-8e95-b56045b67a45 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:48.021217 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea48ff-a610-4e81-badb-10b78f5f14a2 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:48.024006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 381/772] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:48.025376 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32654bf-594f-4654-8e95-b56045b67a45 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:48.065590 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592be3ef-85e5-41bf-a255-56977d199ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:48.066410 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592be3ef-85e5-41bf-a255-56977d199ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:48.066812 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592be3ef-85e5-41bf-a255-56977d199ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:48.067026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592be3ef-85e5-41bf-a255-56977d199ad4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:48.081308 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592be3ef-85e5-41bf-a255-56977d199ad4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:48.083039 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592be3ef-85e5-41bf-a255-56977d199ad4 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:48.099179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-592be3ef-85e5-41bf-a255-56977d199ad4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:48.099179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:48.099179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:48.099179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:48.099179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:48.099179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:48.099179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:48.099179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:48.099179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:48.099179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:48.099179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:48.099179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:48.099179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:48.099179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:48.099179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:48.100177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:48.100177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:48.100177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:48.100177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:48.100177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:48.100177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:48.100177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:48.100177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:48.100177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:48.100177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:48.100177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:48.100177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:48.100177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:48.100177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:48.100177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:48.100177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:48.100177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:48.101132 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:48.101132 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:48.101132 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:48.101132 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:48.101132 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 392/773] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:48.101874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541b206-89bc-4163-81e1-a08ddf300c29 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:48.101874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541b206-89bc-4163-81e1-a08ddf300c29 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:48.102111 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541b206-89bc-4163-81e1-a08ddf300c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:48.108944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:48.123079 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-3541b206-89bc-4163-81e1-a08ddf300c29 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:48.124404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3541b206-89bc-4163-81e1-a08ddf300c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:48.161913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32654bf-594f-4654-8e95-b56045b67a45 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:48.204209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bfc4dc-067d-4b31-b6fc-112890c53c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:48.204865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfc4dc-067d-4b31-b6fc-112890c53c10 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:48.205148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfc4dc-067d-4b31-b6fc-112890c53c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:48.205431 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfc4dc-067d-4b31-b6fc-112890c53c10 None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:48.206319 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bfc4dc-067d-4b31-b6fc-112890c53c10 None admin] RBAC: Adding query filter params (type=network) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:48.206577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 393/774] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:58:48.207274 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541b206-89bc-4163-81e1-a08ddf300c29 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:48.217118 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bfc4dc-067d-4b31-b6fc-112890c53c10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:48.219606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bfc4dc-067d-4b31-b6fc-112890c53c10 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:48.222380 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:48.231565 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 382/775] 10.222.0.93 () {56 vars in 1112 bytes} [Mon Aug 1 14:58:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:48.233227 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bfc4dc-067d-4b31-b6fc-112890c53c10 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:48.348903 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bfc4dc-067d-4b31-b6fc-112890c53c10 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:48.359287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541b206-89bc-4163-81e1-a08ddf300c29 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:48.386139 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:48.404118 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d5da0-3188-4587-a055-1b214cb008d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:48.405074 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8bf184-2b21-43ef-8063-cf5862c6c497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:48.405407 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d5da0-3188-4587-a055-1b214cb008d0 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:48.405880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8bf184-2b21-43ef-8063-cf5862c6c497 None admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:48.405987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d5da0-3188-4587-a055-1b214cb008d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:48.406075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8bf184-2b21-43ef-8063-cf5862c6c497 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:48.406381 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8bf184-2b21-43ef-8063-cf5862c6c497 None admin] PATH_INFO: `/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/3d9be74621d14c94b95ae94947e5abbd/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:48.406483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d5da0-3188-4587-a055-1b214cb008d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:48.407523 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d5da0-3188-4587-a055-1b214cb008d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:48.407944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d5da0-3188-4587-a055-1b214cb008d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:48.408376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8bf184-2b21-43ef-8063-cf5862c6c497 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac9f7c74597642b0acecc956429ac4d8, user_id=3d9be74621d14c94b95ae94947e5abbd, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:48.409364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d5da0-3188-4587-a055-1b214cb008d0 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:48.410053 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8bf184-2b21-43ef-8063-cf5862c6c497 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:48.423190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 383/776] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:48.424040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d5da0-3188-4587-a055-1b214cb008d0 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:48.432071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 394/777] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:58:48 2022] PUT /identity/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/3d9be74621d14c94b95ae94947e5abbd/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:58:48.433270 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8bf184-2b21-43ef-8063-cf5862c6c497 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:48.566808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d5da0-3188-4587-a055-1b214cb008d0 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:48.635261 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee7a4d-292c-4830-819a-2b5f961b58a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:48.637822 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee7a4d-292c-4830-819a-2b5f961b58a8 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:48.641958 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee7a4d-292c-4830-819a-2b5f961b58a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:48.641958 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee7a4d-292c-4830-819a-2b5f961b58a8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:48.641958 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee7a4d-292c-4830-819a-2b5f961b58a8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:48.641958 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee7a4d-292c-4830-819a-2b5f961b58a8 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:48.652030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-7aee7a4d-292c-4830-819a-2b5f961b58a8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 14:58:48.652030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:48.652030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:58:48.652030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:58:48.652030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:58:48.652030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:58:48.652030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:58:48.652030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:58:48.652030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 14:58:48.652030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:48.652030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:58:48.652030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:48.652030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:48.652030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:48.652030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:48.652030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:48.654260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:48.654260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:48.654260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:48.654260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:48.654260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:48.654260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:48.654260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:48.654260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:58:48.654260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:58:48.654260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:58:48.654260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:58:48.654260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:48.654260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:48.654260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:48.654260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:48.654260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:48.656232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:48.656232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:48.656232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:48.656232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:48.656232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:48.656232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:48.656232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:48.656232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:48.656232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:48.656232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:48.656232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:48.656232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:58:48.656232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:58:48.656232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 14:58:48.656232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:48.656232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 384/778] 10.222.0.93 () {56 vars in 1129 bytes} [Mon Aug 1 14:58:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:48.658278 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee7a4d-292c-4830-819a-2b5f961b58a8 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:48.682427 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:48.717734 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:48.726370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8bf184-2b21-43ef-8063-cf5862c6c497 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:48.768434 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97298176-deb4-46db-9e60-83c42c0fb03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:48.769497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97298176-deb4-46db-9e60-83c42c0fb03e None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:48.770782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97298176-deb4-46db-9e60-83c42c0fb03e None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:48.771266 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97298176-deb4-46db-9e60-83c42c0fb03e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:48.772898 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97298176-deb4-46db-9e60-83c42c0fb03e None admin] RBAC: Adding query filter params (interface=public, service_id=57bbd71f95c941b4b58ff63ebf0f5700, region_id=RegionOne) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:48.773770 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97298176-deb4-46db-9e60-83c42c0fb03e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:48.775821 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97298176-deb4-46db-9e60-83c42c0fb03e None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:48.781082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee7a4d-292c-4830-819a-2b5f961b58a8 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:48.785999 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 395/779] 10.222.0.93 () {56 vars in 1251 bytes} [Mon Aug 1 14:58:48 2022] GET /identity/v3/endpoints?service_id=57bbd71f95c941b4b58ff63ebf0f5700&interface=public®ion_id=RegionOne => generated 194 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:48.787661 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab9010-5261-4fe0-b201-f3964f32f596 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:48.788150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab9010-5261-4fe0-b201-f3964f32f596 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:48.788786 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab9010-5261-4fe0-b201-f3964f32f596 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:48.796635 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 396/780] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:48.796635 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab9010-5261-4fe0-b201-f3964f32f596 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:48.831697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf4fe2-65f8-4d7d-89aa-66a32aa90568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:48.832400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf4fe2-65f8-4d7d-89aa-66a32aa90568 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:48.833219 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf4fe2-65f8-4d7d-89aa-66a32aa90568 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:48.833504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf4fe2-65f8-4d7d-89aa-66a32aa90568 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:48.841779 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf4fe2-65f8-4d7d-89aa-66a32aa90568 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:48.843397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf4fe2-65f8-4d7d-89aa-66a32aa90568 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:48.857096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-fdbf4fe2-65f8-4d7d-89aa-66a32aa90568 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:48.857096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:48.857096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:48.857096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:48.857096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:48.857096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:48.857096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:48.857096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:48.857096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:48.857096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:48.857096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:48.857096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:48.857096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:48.857096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:48.857096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:48.858251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:48.858251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:48.858251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:48.858251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:48.858251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:48.858251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:48.858251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:48.858251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:48.858251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:48.858251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:48.858251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:48.858251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:48.858251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:48.858251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:48.858251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:48.858251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:48.858251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:48.859441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:48.859441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:48.859441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:48.859441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:48.859441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:48.859441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:48.859441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:48.859441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:48.859441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 385/781] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:48.859956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2dcd92-13ed-4e51-b269-12ebd3f14ef1 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:48.859956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2dcd92-13ed-4e51-b269-12ebd3f14ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:48.860092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2dcd92-13ed-4e51-b269-12ebd3f14ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:48.880094 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-2c2dcd92-13ed-4e51-b269-12ebd3f14ef1 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:48.881365 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-2c2dcd92-13ed-4e51-b269-12ebd3f14ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:48.931604 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab9010-5261-4fe0-b201-f3964f32f596 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:48.959101 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 386/782] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:58:48.959772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2dcd92-13ed-4e51-b269-12ebd3f14ef1 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:48.981315 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611cb477-d1b3-48f3-b30c-dbb35303ebd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:48.982229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611cb477-d1b3-48f3-b30c-dbb35303ebd7 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:48.982512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611cb477-d1b3-48f3-b30c-dbb35303ebd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:48.982785 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611cb477-d1b3-48f3-b30c-dbb35303ebd7 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:48.983637 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611cb477-d1b3-48f3-b30c-dbb35303ebd7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:48.983945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611cb477-d1b3-48f3-b30c-dbb35303ebd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:48.985694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611cb477-d1b3-48f3-b30c-dbb35303ebd7 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:48.986992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:48.998574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 397/783] 10.222.0.93 () {56 vars in 1124 bytes} [Mon Aug 1 14:58:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:48.998574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611cb477-d1b3-48f3-b30c-dbb35303ebd7 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:49.012829 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:49.086197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2dcd92-13ed-4e51-b269-12ebd3f14ef1 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:49.114346 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611cb477-d1b3-48f3-b30c-dbb35303ebd7 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:49.123783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f6ae7-2ad8-4079-acf0-ad28f09e209c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:49.124416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f6ae7-2ad8-4079-acf0-ad28f09e209c None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:49.124731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f6ae7-2ad8-4079-acf0-ad28f09e209c None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:49.125009 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f6ae7-2ad8-4079-acf0-ad28f09e209c None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:49.125678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f6ae7-2ad8-4079-acf0-ad28f09e209c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:49.127069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f6ae7-2ad8-4079-acf0-ad28f09e209c None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:49.133348 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 387/784] 10.222.0.93 () {56 vars in 1087 bytes} [Mon Aug 1 14:58:48 2022] GET /identity/v3/services => generated 772 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:49.134186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f6ae7-2ad8-4079-acf0-ad28f09e209c None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:49.171739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b76465-91e3-43de-948a-65582579cfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:49.172966 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b76465-91e3-43de-948a-65582579cfd5 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:49.173903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b76465-91e3-43de-948a-65582579cfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:49.174337 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b76465-91e3-43de-948a-65582579cfd5 None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:49.175576 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b76465-91e3-43de-948a-65582579cfd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:49.176043 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b76465-91e3-43de-948a-65582579cfd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:49.179250 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b76465-91e3-43de-948a-65582579cfd5 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:49.202865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 398/785] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:49.202865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b76465-91e3-43de-948a-65582579cfd5 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:49.276113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f6ae7-2ad8-4079-acf0-ad28f09e209c None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:49.280931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:49.292853 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:49.332243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24da62-28db-4eae-9520-034c92f653a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:49.332971 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24da62-28db-4eae-9520-034c92f653a2 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:49.333255 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24da62-28db-4eae-9520-034c92f653a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:49.333534 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24da62-28db-4eae-9520-034c92f653a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:49.348919 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24da62-28db-4eae-9520-034c92f653a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:49.354628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-ed24da62-28db-4eae-9520-034c92f653a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:58:49.354628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:49.354628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:49.354628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:49.354628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:49.354628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:49.354628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:49.354628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:49.354628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:49.354628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:49.354628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:49.354628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:49.354628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:49.354628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:49.354628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:49.354628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:49.355721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:49.355721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:49.355721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:49.355721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:49.355721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:49.355721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:49.355721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:49.355721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:49.355721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:49.355721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:49.355721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:49.355721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:49.355721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:49.355721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:49.355721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:49.355721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:49.360697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:49.360697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:49.360697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:49.360697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:49.360697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:49.360697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:49.360697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:49.360697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:49.360697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:49.360697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:49.360697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:58:49.360697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:49.360697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 388/786] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:58:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:49.360697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24da62-28db-4eae-9520-034c92f653a2 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:49.364528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b76465-91e3-43de-948a-65582579cfd5 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:49.416158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0f01f-5418-482c-8ecc-711111e0bec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:49.417313 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0f01f-5418-482c-8ecc-711111e0bec8 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:49.418079 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0f01f-5418-482c-8ecc-711111e0bec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:49.418536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0f01f-5418-482c-8ecc-711111e0bec8 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:49.419397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0f01f-5418-482c-8ecc-711111e0bec8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:49.419901 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0f01f-5418-482c-8ecc-711111e0bec8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:49.422098 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0f01f-5418-482c-8ecc-711111e0bec8 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:49.434845 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 399/787] 10.222.0.93 () {56 vars in 1124 bytes} [Mon Aug 1 14:58:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:49.434845 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0f01f-5418-482c-8ecc-711111e0bec8 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:49.481713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24da62-28db-4eae-9520-034c92f653a2 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:49.532722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf73cc4-49ee-4b48-916e-0719bfcfe161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:49.533756 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf73cc4-49ee-4b48-916e-0719bfcfe161 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:49.534180 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf73cc4-49ee-4b48-916e-0719bfcfe161 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:49.534593 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf73cc4-49ee-4b48-916e-0719bfcfe161 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:49.536565 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf73cc4-49ee-4b48-916e-0719bfcfe161 None admin] RBAC: Adding query filter params (role.id=6cff92a2d53f4855b38a9d9f11aa19bc, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=4370f8f474c2446e989d0d8c5b75e87a) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:49.537029 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf73cc4-49ee-4b48-916e-0719bfcfe161 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:49.538643 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf73cc4-49ee-4b48-916e-0719bfcfe161 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:49.546513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 389/788] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:58:49 2022] GET /identity/v3/role_assignments?user.id=4370f8f474c2446e989d0d8c5b75e87a&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=6cff92a2d53f4855b38a9d9f11aa19bc => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:49.547614 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf73cc4-49ee-4b48-916e-0719bfcfe161 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:49.608869 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:49.679194 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0f01f-5418-482c-8ecc-711111e0bec8 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:49.709118 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf73cc4-49ee-4b48-916e-0719bfcfe161 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:49.731277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:49.752789 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263ae98-8913-4314-b459-bcc93cdf6da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:49.753338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263ae98-8913-4314-b459-bcc93cdf6da5 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:49.753619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263ae98-8913-4314-b459-bcc93cdf6da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:49.754057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263ae98-8913-4314-b459-bcc93cdf6da5 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:49.754877 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263ae98-8913-4314-b459-bcc93cdf6da5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:49.755185 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263ae98-8913-4314-b459-bcc93cdf6da5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:49.756823 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263ae98-8913-4314-b459-bcc93cdf6da5 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:49.769458 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 400/789] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:58:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:49.777525 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263ae98-8913-4314-b459-bcc93cdf6da5 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:49.777821 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf002f8-37cf-4500-8ec6-792f40afa102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:49.778645 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf002f8-37cf-4500-8ec6-792f40afa102 None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:49.779096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf002f8-37cf-4500-8ec6-792f40afa102 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:49.779524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf002f8-37cf-4500-8ec6-792f40afa102 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:49.780404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf002f8-37cf-4500-8ec6-792f40afa102 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:49.783445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf002f8-37cf-4500-8ec6-792f40afa102 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:49.823495 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 390/790] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 14:58:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:58:49.924974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263ae98-8913-4314-b459-bcc93cdf6da5 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:49.977905 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b696767-c0e6-4928-82e7-85034e868f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:49.979345 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b696767-c0e6-4928-82e7-85034e868f37 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:49.979877 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b696767-c0e6-4928-82e7-85034e868f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:49.980332 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b696767-c0e6-4928-82e7-85034e868f37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:49.994694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b696767-c0e6-4928-82e7-85034e868f37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:49.996933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b696767-c0e6-4928-82e7-85034e868f37 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:50.016504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-5b696767-c0e6-4928-82e7-85034e868f37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:50.016504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:50.016504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:50.016504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:50.016504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:50.016504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:50.016504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:50.016504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:50.016504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:50.016504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:50.016504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:50.016504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:50.016504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:50.016504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:50.016504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:50.017756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:50.017756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:50.017756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:50.017756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:50.017756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:50.017756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:50.017756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:50.017756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:50.017756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:50.017756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:50.017756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:50.017756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:50.017756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:50.017756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:50.017756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:50.017756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:50.017756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:50.019276 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:50.019276 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:50.019276 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:50.019276 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:50.022285 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 401/791] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:58:50.031365 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf002f8-37cf-4500-8ec6-792f40afa102 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:50.179462 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:50.206326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf002f8-37cf-4500-8ec6-792f40afa102 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:50.264095 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3dbce-4788-4c0e-8b2e-ac297729572a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:50.264800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3dbce-4788-4c0e-8b2e-ac297729572a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:50.265082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3dbce-4788-4c0e-8b2e-ac297729572a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:50.265357 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3dbce-4788-4c0e-8b2e-ac297729572a None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:50.266067 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3dbce-4788-4c0e-8b2e-ac297729572a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:50.266366 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3dbce-4788-4c0e-8b2e-ac297729572a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:50.267867 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3dbce-4788-4c0e-8b2e-ac297729572a None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:50.283038 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 391/792] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:50.290275 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b696767-c0e6-4928-82e7-85034e868f37 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:50.328278 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:50.409922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b696767-c0e6-4928-82e7-85034e868f37 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:50.451760 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ecaad-d0fa-43d3-a602-cdba3205aa52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:50.452417 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ecaad-d0fa-43d3-a602-cdba3205aa52 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:50.452720 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ecaad-d0fa-43d3-a602-cdba3205aa52 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:50.452995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ecaad-d0fa-43d3-a602-cdba3205aa52 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:50.465341 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ecaad-d0fa-43d3-a602-cdba3205aa52 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:50.466811 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ecaad-d0fa-43d3-a602-cdba3205aa52 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:50.483771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-ed9ecaad-d0fa-43d3-a602-cdba3205aa52 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:58:50.483771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:50.483771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:50.483771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:50.483771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:50.483771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:50.483771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:50.483771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:50.483771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:50.483771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:50.483771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:50.483771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:50.483771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:50.483771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:50.483771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:50.483771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:50.484815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:50.484815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:50.484815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:50.484815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:50.484815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:50.484815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:50.484815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:50.484815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:50.484815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:50.484815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:50.484815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:50.484815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:50.484815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:50.484815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:50.484815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:50.484815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:50.484815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:50.484815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:50.485813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:50.485813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:50.485813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:50.485813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:50.485813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:50.485813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:50.485813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:50.485813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:50.485813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:58:50.485813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:50.485813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 402/793] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:58:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:50.497377 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3dbce-4788-4c0e-8b2e-ac297729572a None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:50.670812 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3dbce-4788-4c0e-8b2e-ac297729572a None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:50.729285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ae0ba-5fd5-4d49-9f12-b07d356b7225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:50.729937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ae0ba-5fd5-4d49-9f12-b07d356b7225 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:50.730224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ae0ba-5fd5-4d49-9f12-b07d356b7225 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:50.730573 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ae0ba-5fd5-4d49-9f12-b07d356b7225 None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:50.731396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ae0ba-5fd5-4d49-9f12-b07d356b7225 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:50.731709 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ae0ba-5fd5-4d49-9f12-b07d356b7225 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:50.733220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ae0ba-5fd5-4d49-9f12-b07d356b7225 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:50.763375 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 392/794] 10.222.0.93 () {56 vars in 1135 bytes} [Mon Aug 1 14:58:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:50.768580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ecaad-d0fa-43d3-a602-cdba3205aa52 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:50.937388 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:50.948522 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ecaad-d0fa-43d3-a602-cdba3205aa52 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:51.012660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a4b82-2b87-4b69-b3a0-f14ac60befbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:51.013342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a4b82-2b87-4b69-b3a0-f14ac60befbb None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:51.013627 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a4b82-2b87-4b69-b3a0-f14ac60befbb None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:51.013906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a4b82-2b87-4b69-b3a0-f14ac60befbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:51.031751 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a4b82-2b87-4b69-b3a0-f14ac60befbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:51.033552 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa2c6d-5c76-4d7a-9ce9-3b6f9e94296b None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:51.033837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa2c6d-5c76-4d7a-9ce9-3b6f9e94296b None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:51.034112 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa2c6d-5c76-4d7a-9ce9-3b6f9e94296b None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:51.036079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 393/795] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:58:51.037953 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a4b82-2b87-4b69-b3a0-f14ac60befbb None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:51.042159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657a4b8-7ccf-43cc-87a7-33176c98993a None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:51.042577 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657a4b8-7ccf-43cc-87a7-33176c98993a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:51.042966 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657a4b8-7ccf-43cc-87a7-33176c98993a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:51.058154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-434a4b82-2b87-4b69-b3a0-f14ac60befbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:51.058154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:51.058154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:51.058154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:51.058154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:51.058154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:51.058154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:51.058154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:51.058154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:51.058154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:51.058154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:51.058154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:51.058154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:51.058154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:51.058154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:51.059138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:51.059138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:51.059138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:51.059138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:51.059138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:51.059138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:51.059138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:51.059138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:51.059138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:51.059138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:51.059138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:51.059138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:51.059138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:51.059138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:51.059138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:51.059138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:51.059138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:51.060063 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:51.060063 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:51.060063 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:51.060063 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:51.064259 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 403/796] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:58:51.065694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a4b82-2b87-4b69-b3a0-f14ac60befbb None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:51.069847 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:51.093667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-6657a4b8-7ccf-43cc-87a7-33176c98993a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:51.094979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-6657a4b8-7ccf-43cc-87a7-33176c98993a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:51.271644 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a4b82-2b87-4b69-b3a0-f14ac60befbb None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:51.272263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 394/797] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:58:51.337110 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657a4b8-7ccf-43cc-87a7-33176c98993a None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:51.354754 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbfd2e-a71d-434a-9ada-2b92b0e64f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:51.355383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbfd2e-a71d-434a-9ada-2b92b0e64f91 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:51.355662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbfd2e-a71d-434a-9ada-2b92b0e64f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:51.355931 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbfd2e-a71d-434a-9ada-2b92b0e64f91 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:51.356649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cbfd2e-a71d-434a-9ada-2b92b0e64f91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:51.356955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cbfd2e-a71d-434a-9ada-2b92b0e64f91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:51.358349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cbfd2e-a71d-434a-9ada-2b92b0e64f91 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:51.371776 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 404/798] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:51.377465 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbfd2e-a71d-434a-9ada-2b92b0e64f91 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:51.389206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:51.486569 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbfd2e-a71d-434a-9ada-2b92b0e64f91 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:51.528911 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a5ff6-c8d9-4132-9b46-10dd3a5fd696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:51.529844 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a5ff6-c8d9-4132-9b46-10dd3a5fd696 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:51.530314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a5ff6-c8d9-4132-9b46-10dd3a5fd696 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:51.530923 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a5ff6-c8d9-4132-9b46-10dd3a5fd696 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:51.539336 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a5ff6-c8d9-4132-9b46-10dd3a5fd696 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:51.541350 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a5ff6-c8d9-4132-9b46-10dd3a5fd696 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:51.561593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-f21a5ff6-c8d9-4132-9b46-10dd3a5fd696 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:51.561593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:51.561593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:51.561593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:51.561593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:51.561593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:51.561593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:51.561593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:51.561593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:51.561593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:51.561593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:51.561593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:51.561593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:51.561593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:51.561593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:51.562581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:51.562581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:51.562581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:51.562581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:51.562581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:51.562581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:51.562581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:51.562581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:51.562581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:51.562581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:51.562581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:51.562581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:51.562581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:51.562581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:51.562581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:51.562581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:51.562581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:51.563477 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:51.563477 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:51.563477 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:51.563477 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:51.563477 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:51.563477 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:51.563477 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:51.563477 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:51.571452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 405/799] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:58:51.581022 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a5ff6-c8d9-4132-9b46-10dd3a5fd696 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:51.606221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657a4b8-7ccf-43cc-87a7-33176c98993a None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:51.672888 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:51.685736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559e773-d9bb-4b1f-8daf-3f7e774fc633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:51.686405 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559e773-d9bb-4b1f-8daf-3f7e774fc633 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:51.688275 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559e773-d9bb-4b1f-8daf-3f7e774fc633 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:51.688588 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559e773-d9bb-4b1f-8daf-3f7e774fc633 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:51.717789 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559e773-d9bb-4b1f-8daf-3f7e774fc633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:51.722945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-e559e773-d9bb-4b1f-8daf-3f7e774fc633 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:58:51.722945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:51.722945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:51.722945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:51.722945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:51.722945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:51.722945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:51.722945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:51.722945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:51.722945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:51.722945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:51.722945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:51.722945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:51.722945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:51.722945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:51.722945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:51.723993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:51.723993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:51.723993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:51.723993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:51.723993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:51.723993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:51.723993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:51.723993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:51.723993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:51.723993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:51.723993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:51.723993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:51.723993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:51.723993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:51.723993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:51.723993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:51.724965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:51.724965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:51.724965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:51.724965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:51.724965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:51.724965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:51.724965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:51.724965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:51.724965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:51.724965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:51.724965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:58:51.724965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:51.730521 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:51.735082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 395/800] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:58:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:51.736038 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559e773-d9bb-4b1f-8daf-3f7e774fc633 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:51.790975 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a5ff6-c8d9-4132-9b46-10dd3a5fd696 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:51.844214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb012b4-f30b-4295-a11f-bc465b10cd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:51.845055 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb012b4-f30b-4295-a11f-bc465b10cd70 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:51.845335 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb012b4-f30b-4295-a11f-bc465b10cd70 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:51.845604 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb012b4-f30b-4295-a11f-bc465b10cd70 None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:51.846409 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb012b4-f30b-4295-a11f-bc465b10cd70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:51.846713 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb012b4-f30b-4295-a11f-bc465b10cd70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:51.848836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb012b4-f30b-4295-a11f-bc465b10cd70 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:51.868749 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 406/801] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:51.870614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e99f9-bbec-4060-ae4a-e748b797d028 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:51.870858 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e99f9-bbec-4060-ae4a-e748b797d028 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:51.871136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e99f9-bbec-4060-ae4a-e748b797d028 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:51.878511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 407/802] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:51 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:51.880209 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e99f9-bbec-4060-ae4a-e748b797d028 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:51.885669 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559e773-d9bb-4b1f-8daf-3f7e774fc633 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:51.925410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8008a-1a46-44e9-ba2e-add3c5b15570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:51.926495 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8008a-1a46-44e9-ba2e-add3c5b15570 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:51.926995 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8008a-1a46-44e9-ba2e-add3c5b15570 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:51.927541 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8008a-1a46-44e9-ba2e-add3c5b15570 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:51.928584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee8008a-1a46-44e9-ba2e-add3c5b15570 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:51.929121 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee8008a-1a46-44e9-ba2e-add3c5b15570 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:51.930894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee8008a-1a46-44e9-ba2e-add3c5b15570 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:51.941288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 396/803] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:58:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:51.943630 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c136b94-1b31-4e0c-a754-159e07c6e6f9 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:51.944179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c136b94-1b31-4e0c-a754-159e07c6e6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:51.944712 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c136b94-1b31-4e0c-a754-159e07c6e6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:51.964985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-2c136b94-1b31-4e0c-a754-159e07c6e6f9 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:51.966508 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-2c136b94-1b31-4e0c-a754-159e07c6e6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:52.004104 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e99f9-bbec-4060-ae4a-e748b797d028 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:52.027068 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:52.036262 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:52.049740 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d88a3-2ef9-4e56-8d3e-db61d1d20ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:52.050447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d88a3-2ef9-4e56-8d3e-db61d1d20ed3 None admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:52.050654 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d88a3-2ef9-4e56-8d3e-db61d1d20ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:52.050875 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d88a3-2ef9-4e56-8d3e-db61d1d20ed3 None admin] PATH_INFO: `/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/4df37b02152c48509243b73ebbf49d8d/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:52.051065 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 397/804] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:58:52.052678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c136b94-1b31-4e0c-a754-159e07c6e6f9 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:52.057886 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d88a3-2ef9-4e56-8d3e-db61d1d20ed3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac9f7c74597642b0acecc956429ac4d8, user_id=4df37b02152c48509243b73ebbf49d8d, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:52.059586 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d88a3-2ef9-4e56-8d3e-db61d1d20ed3 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:52.090136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 408/805] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:58:51 2022] PUT /identity/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/4df37b02152c48509243b73ebbf49d8d/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:58:52.096767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d88a3-2ef9-4e56-8d3e-db61d1d20ed3 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:52.277568 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c136b94-1b31-4e0c-a754-159e07c6e6f9 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:52.328238 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d88a3-2ef9-4e56-8d3e-db61d1d20ed3 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:52.338186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e0e48-2313-43ac-820d-9324c11b55b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:52.339938 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e0e48-2313-43ac-820d-9324c11b55b6 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:52.340594 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e0e48-2313-43ac-820d-9324c11b55b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:52.341109 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e0e48-2313-43ac-820d-9324c11b55b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:52.356398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e0e48-2313-43ac-820d-9324c11b55b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:52.357906 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e0e48-2313-43ac-820d-9324c11b55b6 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:52.374079 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621102de-2e44-411f-9908-82120f5bde21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:52.375088 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621102de-2e44-411f-9908-82120f5bde21 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:52.375561 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621102de-2e44-411f-9908-82120f5bde21 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:52.376034 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621102de-2e44-411f-9908-82120f5bde21 None admin] PATH_INFO: `/v3/services/network` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:52.376698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-917e0e48-2313-43ac-820d-9324c11b55b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:52.376698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:52.376698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:52.376698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:52.376698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:52.376698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:52.376698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:52.376698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:52.376698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:52.376698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:52.376698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:52.376698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:52.376698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:52.376698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:52.376698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:52.377639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:52.377639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:52.377639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:52.377639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:52.377639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:52.377639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:52.377639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:52.377639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:52.377639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:52.377639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:52.377639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:52.377639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:52.377639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:52.377639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:52.377639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:52.377639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:52.377639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:52.378751 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:52.378751 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:52.378751 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:52.378751 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:52.378751 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 398/806] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:52.379983 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb841b-a114-4048-8ad6-33c15d1f4232 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:52.380224 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621102de-2e44-411f-9908-82120f5bde21 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:52.380335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb841b-a114-4048-8ad6-33c15d1f4232 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:52.382076 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621102de-2e44-411f-9908-82120f5bde21 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:52.385545 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb841b-a114-4048-8ad6-33c15d1f4232 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:52.387457 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 399/807] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:52.388353 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb841b-a114-4048-8ad6-33c15d1f4232 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:52.394011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-621102de-2e44-411f-9908-82120f5bde21 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 14:58:52.394011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:52.394011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:58:52.394011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:58:52.394011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:58:52.394011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:58:52.394011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:58:52.394011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:58:52.394011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 14:58:52.394011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:52.394011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:58:52.394011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:52.394011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:52.394011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:52.394011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:52.394011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:52.394011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:52.395946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:52.395946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:52.395946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:52.395946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:52.395946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:52.395946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:52.395946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:58:52.395946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:58:52.395946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:58:52.395946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:58:52.395946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:52.395946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:52.395946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:52.395946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:52.395946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:52.395946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:52.396986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:52.396986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:52.396986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:52.396986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:52.396986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:52.396986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:52.396986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:52.396986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:52.396986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:52.396986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:52.396986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:58:52.396986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:58:52.396986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 14:58:52.396986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:52.399506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 409/808] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:52 2022] GET /identity/v3/services/network => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:52.406702 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621102de-2e44-411f-9908-82120f5bde21 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:52.433012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:52.466460 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:52.510829 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb841b-a114-4048-8ad6-33c15d1f4232 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:52.528526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621102de-2e44-411f-9908-82120f5bde21 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:52.564188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd066c-0ff5-417d-a618-a9e754a2fd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:52.565136 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd066c-0ff5-417d-a618-a9e754a2fd44 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:52.565587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd066c-0ff5-417d-a618-a9e754a2fd44 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:52.566009 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd066c-0ff5-417d-a618-a9e754a2fd44 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:52.567547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd066c-0ff5-417d-a618-a9e754a2fd44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:52.568103 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd066c-0ff5-417d-a618-a9e754a2fd44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:52.569033 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b379c76-52e9-4c10-a442-10094a978a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:52.569768 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b379c76-52e9-4c10-a442-10094a978a28 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:52.570123 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b379c76-52e9-4c10-a442-10094a978a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:52.570364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd066c-0ff5-417d-a618-a9e754a2fd44 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:52.570533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b379c76-52e9-4c10-a442-10094a978a28 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:52.571314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b379c76-52e9-4c10-a442-10094a978a28 None admin] RBAC: Adding query filter params (name=network) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:52.571695 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b379c76-52e9-4c10-a442-10094a978a28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:52.573227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b379c76-52e9-4c10-a442-10094a978a28 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:52.584019 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 410/809] 10.222.0.93 () {56 vars in 1112 bytes} [Mon Aug 1 14:58:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:52.584019 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22df04-09be-4b4c-92c7-a8777f475db0 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:52.584019 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22df04-09be-4b4c-92c7-a8777f475db0 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:52.584019 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22df04-09be-4b4c-92c7-a8777f475db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:52.596654 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 400/810] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:52.597650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd066c-0ff5-417d-a618-a9e754a2fd44 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:52.605719 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-3b22df04-09be-4b4c-92c7-a8777f475db0 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:52.607941 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3b22df04-09be-4b4c-92c7-a8777f475db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:52.705198 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 411/811] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:58:52.706908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22df04-09be-4b4c-92c7-a8777f475db0 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:52.719229 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:52.727024 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd066c-0ff5-417d-a618-a9e754a2fd44 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:52.769542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140105b-b425-450d-be86-28e4773d26a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:52.770754 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140105b-b425-450d-be86-28e4773d26a2 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:52.771088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140105b-b425-450d-be86-28e4773d26a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:52.771483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140105b-b425-450d-be86-28e4773d26a2 None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:52.772318 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:52.772708 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140105b-b425-450d-be86-28e4773d26a2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:52.773023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140105b-b425-450d-be86-28e4773d26a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:52.774569 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140105b-b425-450d-be86-28e4773d26a2 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:52.781910 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 401/812] 10.222.0.93 () {56 vars in 1112 bytes} [Mon Aug 1 14:58:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:52.782761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140105b-b425-450d-be86-28e4773d26a2 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:52.847885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22df04-09be-4b4c-92c7-a8777f475db0 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:52.891466 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dd0e3-efe8-4278-be50-9f0dc41e9d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:52.892749 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dd0e3-efe8-4278-be50-9f0dc41e9d42 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:52.892749 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dd0e3-efe8-4278-be50-9f0dc41e9d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:52.893031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dd0e3-efe8-4278-be50-9f0dc41e9d42 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:52.902068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30dd0e3-efe8-4278-be50-9f0dc41e9d42 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:52.903696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30dd0e3-efe8-4278-be50-9f0dc41e9d42 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:52.917849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-d30dd0e3-efe8-4278-be50-9f0dc41e9d42 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:58:52.917849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:52.917849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:52.917849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:52.917849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:52.917849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:52.917849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:52.917849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:52.917849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:52.917849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:52.917849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:52.917849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:52.917849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:52.917849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:52.917849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:52.917849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:52.918899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:52.918899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:52.918899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:52.918899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:52.918899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:52.918899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:52.918899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:52.918899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:52.918899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:52.918899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:52.918899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:52.918899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:52.918899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:52.918899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:52.918899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:52.918899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:52.918899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:52.918899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:52.919865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:52.919865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:52.919865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:52.919865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:52.919865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:52.919865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:52.919865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:52.919865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:52.919865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:58:52.919865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:52.921120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 412/813] 10.222.0.93 () {56 vars in 1093 bytes} [Mon Aug 1 14:58:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:52.922191 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dd0e3-efe8-4278-be50-9f0dc41e9d42 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:52.939282 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140105b-b425-450d-be86-28e4773d26a2 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:52.993713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d36f9-8f51-42ba-a8b4-159950989099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:52.994358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d36f9-8f51-42ba-a8b4-159950989099 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:52.994643 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d36f9-8f51-42ba-a8b4-159950989099 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:52.994919 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d36f9-8f51-42ba-a8b4-159950989099 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:53.009753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d36f9-8f51-42ba-a8b4-159950989099 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:53.017416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-996d36f9-8f51-42ba-a8b4-159950989099 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:53.017416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:53.017416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:53.017416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:53.017416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:53.017416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:53.017416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:53.017416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:53.017416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:53.017416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:53.017416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:53.017416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:53.017416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:53.017416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:53.017416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:53.017416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:53.018826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:53.018826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:53.018826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:53.018826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:53.018826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:53.018826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:53.018826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:53.018826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:53.018826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:53.018826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:53.018826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:53.018826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:53.018826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:53.018826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:53.018826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:53.018826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:53.021127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:53.021127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:53.021127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:53.021127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:53.021127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:53.021127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:53.021127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:53.021127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:53.021127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:53.021127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:53.021127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:53.021127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:53.021127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 402/814] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:53.021127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b8528-bc8c-4550-ad45-628a77e9e791 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:53.022266 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b8528-bc8c-4550-ad45-628a77e9e791 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:53.022266 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b8528-bc8c-4550-ad45-628a77e9e791 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:53.027286 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 403/815] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:53.028151 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b8528-bc8c-4550-ad45-628a77e9e791 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:53.059026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dd0e3-efe8-4278-be50-9f0dc41e9d42 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:53.060888 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:53.062325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:53.109692 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd795f-70a4-4363-827c-9dd8570b8ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:53.110206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd795f-70a4-4363-827c-9dd8570b8ee6 None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:53.110514 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd795f-70a4-4363-827c-9dd8570b8ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:53.110760 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd795f-70a4-4363-827c-9dd8570b8ee6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:53.111473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd795f-70a4-4363-827c-9dd8570b8ee6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:53.112883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd795f-70a4-4363-827c-9dd8570b8ee6 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:53.152506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 413/816] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 14:58:52 2022] POST /identity/v3/endpoints => generated 327 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:58:53.152506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd795f-70a4-4363-827c-9dd8570b8ee6 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:53.201315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b8528-bc8c-4550-ad45-628a77e9e791 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:53.243108 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f7b56-49a8-4d96-a48d-9fc39cfafc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:53.243846 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f7b56-49a8-4d96-a48d-9fc39cfafc9e None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:53.244125 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f7b56-49a8-4d96-a48d-9fc39cfafc9e None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:53.244391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f7b56-49a8-4d96-a48d-9fc39cfafc9e None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:53.246510 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f7b56-49a8-4d96-a48d-9fc39cfafc9e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:53.246510 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f7b56-49a8-4d96-a48d-9fc39cfafc9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:53.247283 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f7b56-49a8-4d96-a48d-9fc39cfafc9e None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:53.276194 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 404/817] 10.222.0.93 () {56 vars in 1133 bytes} [Mon Aug 1 14:58:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:53.279043 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e267f-2a2c-4216-9c6e-7034fe49a9dc None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:53.279811 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e267f-2a2c-4216-9c6e-7034fe49a9dc None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:53.280765 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e267f-2a2c-4216-9c6e-7034fe49a9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:53.312982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-306e267f-2a2c-4216-9c6e-7034fe49a9dc None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:53.315555 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd795f-70a4-4363-827c-9dd8570b8ee6 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:53.316168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-306e267f-2a2c-4216-9c6e-7034fe49a9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:53.358220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c781a7-8013-407b-ab86-4196956d9538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:53.358833 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c781a7-8013-407b-ab86-4196956d9538 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:53.359140 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c781a7-8013-407b-ab86-4196956d9538 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:53.359388 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c781a7-8013-407b-ab86-4196956d9538 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:53.360176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c781a7-8013-407b-ab86-4196956d9538 None admin] RBAC: Adding query filter params (name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:53.360437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c781a7-8013-407b-ab86-4196956d9538 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:53.361852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c781a7-8013-407b-ab86-4196956d9538 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:53.374293 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 414/818] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 14:58:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:53.374543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c781a7-8013-407b-ab86-4196956d9538 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:53.408610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:53.449281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 405/819] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:53 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:58:53.450438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e267f-2a2c-4216-9c6e-7034fe49a9dc None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:53.456456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:53.543347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c781a7-8013-407b-ab86-4196956d9538 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:53.585629 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e267f-2a2c-4216-9c6e-7034fe49a9dc None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:53.587692 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736cb32-c128-4e45-b975-d4cb07175b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:53.588278 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736cb32-c128-4e45-b975-d4cb07175b04 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:53.589609 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736cb32-c128-4e45-b975-d4cb07175b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:53.589609 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736cb32-c128-4e45-b975-d4cb07175b04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:53.627541 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399738d9-e2eb-4a2b-8a55-faa175ebda44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:53.628429 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399738d9-e2eb-4a2b-8a55-faa175ebda44 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:53.628655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399738d9-e2eb-4a2b-8a55-faa175ebda44 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:53.629027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399738d9-e2eb-4a2b-8a55-faa175ebda44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:53.640712 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399738d9-e2eb-4a2b-8a55-faa175ebda44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:53.642347 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399738d9-e2eb-4a2b-8a55-faa175ebda44 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:53.658391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-399738d9-e2eb-4a2b-8a55-faa175ebda44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:53.658391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:53.658391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:53.658391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:53.658391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:53.658391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:53.658391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:53.658391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:53.658391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:53.658391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:53.658391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:53.658391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:53.658391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:53.658391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:53.658391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:53.659542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:53.659542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:53.659542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:53.659542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:53.659542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:53.659542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:53.659542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:53.659542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:53.659542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:53.659542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:53.659542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:53.659542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:53.659542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:53.659542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:53.659542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:53.659542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:53.659542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:53.660889 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:53.660889 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:53.660889 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:53.660889 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:53.663092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 406/820] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:53.664428 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399738d9-e2eb-4a2b-8a55-faa175ebda44 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:53.679710 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736cb32-c128-4e45-b975-d4cb07175b04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:53.681414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736cb32-c128-4e45-b975-d4cb07175b04 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:53.707444 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:53.734142 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:53.788326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399738d9-e2eb-4a2b-8a55-faa175ebda44 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:53.816697 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 415/821] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:53 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:58:53.818753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736cb32-c128-4e45-b975-d4cb07175b04 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:53.839769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fef08-8347-4883-ad26-42a509721b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:53.839769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fef08-8347-4883-ad26-42a509721b8d None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:53.840636 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fef08-8347-4883-ad26-42a509721b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:53.841089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fef08-8347-4883-ad26-42a509721b8d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:53.841581 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fef08-8347-4883-ad26-42a509721b8d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:53.843050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fef08-8347-4883-ad26-42a509721b8d None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:53.853438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-363fef08-8347-4883-ad26-42a509721b8d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 14:58:53.853438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:53.853438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:58:53.853438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:58:53.853438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:58:53.853438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:58:53.853438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:58:53.853438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:58:53.853438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 14:58:53.853438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:53.853438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:58:53.853438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:53.853438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:53.853438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:53.853438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:53.853438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:53.853438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:53.855361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:53.855361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:53.855361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:53.855361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:53.855361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:53.855361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:53.855361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:58:53.855361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:58:53.855361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:58:53.855361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:58:53.855361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:53.855361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:53.855361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:53.855361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:53.855361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:53.855361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:53.857265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:53.857265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:53.857265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:53.857265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:53.857265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:53.857265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:53.857265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:53.857265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:53.857265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:53.857265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:53.857265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:58:53.857265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:58:53.857265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 14:58:53.857265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:53.857265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 407/822] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:58:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:53.859210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fef08-8347-4883-ad26-42a509721b8d None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:53.968567 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736cb32-c128-4e45-b975-d4cb07175b04 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:53.992717 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:54.015103 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fef08-8347-4883-ad26-42a509721b8d None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:54.025880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba4170-5577-436a-8ee9-01fb7b0ea178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:54.027506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba4170-5577-436a-8ee9-01fb7b0ea178 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:54.028500 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba4170-5577-436a-8ee9-01fb7b0ea178 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:54.029367 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba4170-5577-436a-8ee9-01fb7b0ea178 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:54.030584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba4170-5577-436a-8ee9-01fb7b0ea178 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:54.030974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba4170-5577-436a-8ee9-01fb7b0ea178 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:54.033704 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba4170-5577-436a-8ee9-01fb7b0ea178 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:54.039893 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:54.053233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 416/823] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:54.059406 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba4170-5577-436a-8ee9-01fb7b0ea178 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:54.065753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ece6d-b942-4773-8288-b4aa28a9b1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:54.066730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ece6d-b942-4773-8288-b4aa28a9b1aa None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:54.067223 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ece6d-b942-4773-8288-b4aa28a9b1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:54.067519 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ece6d-b942-4773-8288-b4aa28a9b1aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:54.081940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ece6d-b942-4773-8288-b4aa28a9b1aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:54.083460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ece6d-b942-4773-8288-b4aa28a9b1aa None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:54.101674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-095ece6d-b942-4773-8288-b4aa28a9b1aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:54.101674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:54.101674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:54.101674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:54.101674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:54.101674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:54.101674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:54.101674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:54.101674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:54.101674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:54.101674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:54.101674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:54.101674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:54.101674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:54.101674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:54.103115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:54.103115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:54.103115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:54.103115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:54.103115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:54.103115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:54.103115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:54.103115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:54.103115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:54.103115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:54.103115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:54.103115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:54.103115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:54.103115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:54.103115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:54.103115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:54.103115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:54.104940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:54.104940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:54.104940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:54.104940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:54.104940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 408/824] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:54.104940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ece6d-b942-4773-8288-b4aa28a9b1aa None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:54.214439 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba4170-5577-436a-8ee9-01fb7b0ea178 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:54.236848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ece6d-b942-4773-8288-b4aa28a9b1aa None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:54.266094 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d268f-4e02-446c-ae79-76074c19f3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:54.267602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d268f-4e02-446c-ae79-76074c19f3cc None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:54.268133 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d268f-4e02-446c-ae79-76074c19f3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:54.268771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d268f-4e02-446c-ae79-76074c19f3cc None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:54.270596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934d268f-4e02-446c-ae79-76074c19f3cc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:54.271352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934d268f-4e02-446c-ae79-76074c19f3cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:54.273222 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934d268f-4e02-446c-ae79-76074c19f3cc None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:54.279761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3c833-b258-4441-9e23-ab36b05244a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:54.280551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3c833-b258-4441-9e23-ab36b05244a6 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:54.280951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3c833-b258-4441-9e23-ab36b05244a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:54.281164 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3c833-b258-4441-9e23-ab36b05244a6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:54.283398 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 417/825] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:58:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:54.285599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d268f-4e02-446c-ae79-76074c19f3cc None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:54.291311 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3c833-b258-4441-9e23-ab36b05244a6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:54.292883 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3c833-b258-4441-9e23-ab36b05244a6 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:54.306037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-73f3c833-b258-4441-9e23-ab36b05244a6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:58:54.306037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:54.306037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:54.306037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:54.306037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:54.306037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:54.306037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:54.306037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:54.306037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:54.306037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:54.306037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:54.306037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:54.306037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:54.306037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:54.306037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:54.306037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:54.307072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:54.307072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:54.307072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:54.307072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:54.307072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:54.307072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:54.307072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:54.307072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:54.307072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:54.307072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:54.307072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:54.307072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:54.307072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:54.307072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:54.307072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:54.307072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:54.307072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:54.307072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:54.308028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:54.308028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:54.308028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:54.308028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:54.308028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:54.308028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:54.308028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:54.308028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:54.308028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:58:54.308028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:54.308028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 409/826] 10.222.0.93 () {56 vars in 1117 bytes} [Mon Aug 1 14:58:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:54.309309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3c833-b258-4441-9e23-ab36b05244a6 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:54.354626 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:54.359107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:54.437091 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d268f-4e02-446c-ae79-76074c19f3cc None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:54.449656 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3c833-b258-4441-9e23-ab36b05244a6 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:54.508092 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb47d99-6dbc-4996-8c2d-d32236301d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:54.509212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb47d99-6dbc-4996-8c2d-d32236301d2f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:54.509569 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb47d99-6dbc-4996-8c2d-d32236301d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:54.510067 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb47d99-6dbc-4996-8c2d-d32236301d2f None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:54.511248 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb47d99-6dbc-4996-8c2d-d32236301d2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:54.511809 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb47d99-6dbc-4996-8c2d-d32236301d2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:54.514320 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb47d99-6dbc-4996-8c2d-d32236301d2f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:54.523852 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d3295-756c-4572-81dd-edf4c11fe714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:54.524603 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d3295-756c-4572-81dd-edf4c11fe714 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:54.524905 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d3295-756c-4572-81dd-edf4c11fe714 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:54.525063 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d3295-756c-4572-81dd-edf4c11fe714 None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:54.525976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d3295-756c-4572-81dd-edf4c11fe714 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:54.526752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d3295-756c-4572-81dd-edf4c11fe714 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:54.528107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d3295-756c-4572-81dd-edf4c11fe714 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:54.533281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 410/827] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:58:54 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:54.534164 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d3295-756c-4572-81dd-edf4c11fe714 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:54.538823 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 418/828] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:54.541585 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3dec1-4b97-427d-a3f1-be098908192c None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:54.542086 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3dec1-4b97-427d-a3f1-be098908192c None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:54.542609 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3dec1-4b97-427d-a3f1-be098908192c None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:54.548664 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 419/829] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:54.548664 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3dec1-4b97-427d-a3f1-be098908192c None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:54.666812 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:54.674580 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d3295-756c-4572-81dd-edf4c11fe714 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:54.715746 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3dec1-4b97-427d-a3f1-be098908192c None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:54.718447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:54.721249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa7edd-9502-4961-9d10-b58fd3725b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:54.721871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa7edd-9502-4961-9d10-b58fd3725b10 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:54.722152 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa7edd-9502-4961-9d10-b58fd3725b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:54.722426 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa7edd-9502-4961-9d10-b58fd3725b10 None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:54.723522 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa7edd-9502-4961-9d10-b58fd3725b10 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:54.723660 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa7edd-9502-4961-9d10-b58fd3725b10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:54.725072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa7edd-9502-4961-9d10-b58fd3725b10 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:54.745612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 411/830] 10.222.0.93 () {56 vars in 1149 bytes} [Mon Aug 1 14:58:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:54.747721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0f8c3-2c1c-431f-be09-b6dd2df5b4ae None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:54.747721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0f8c3-2c1c-431f-be09-b6dd2df5b4ae None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:54.747721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0f8c3-2c1c-431f-be09-b6dd2df5b4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:54.770569 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-b9a0f8c3-2c1c-431f-be09-b6dd2df5b4ae None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:54.771903 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b9a0f8c3-2c1c-431f-be09-b6dd2df5b4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:54.777692 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec6be6-f5f2-4f64-b931-3684f3ffaa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:54.778436 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec6be6-f5f2-4f64-b931-3684f3ffaa1a None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:54.778817 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec6be6-f5f2-4f64-b931-3684f3ffaa1a None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:54.779162 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec6be6-f5f2-4f64-b931-3684f3ffaa1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:54.788288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec6be6-f5f2-4f64-b931-3684f3ffaa1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:54.789826 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec6be6-f5f2-4f64-b931-3684f3ffaa1a None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:54.802988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-70ec6be6-f5f2-4f64-b931-3684f3ffaa1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:54.802988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:54.802988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:54.802988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:54.802988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:54.802988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:54.802988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:54.802988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:54.802988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:54.802988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:54.802988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:54.802988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:54.802988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:54.802988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:54.802988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:54.804328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:54.804328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:54.804328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:54.804328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:54.804328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:54.804328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:54.804328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:54.804328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:54.804328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:54.804328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:54.804328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:54.804328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:54.804328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:54.804328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:54.804328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:54.804328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:54.804328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:54.805797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:54.805797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:54.805797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:54.805797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:54.805797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:54.805797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:54.805797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:54.805797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:54.808835 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 420/831] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:54.810020 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec6be6-f5f2-4f64-b931-3684f3ffaa1a None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:54.884512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 412/832] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:58:54.886303 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0f8c3-2c1c-431f-be09-b6dd2df5b4ae None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:54.964121 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec6be6-f5f2-4f64-b931-3684f3ffaa1a None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:55.027226 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf4b14-c4d5-4270-9465-faea07b1363b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:55.028755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf4b14-c4d5-4270-9465-faea07b1363b None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:55.029075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf4b14-c4d5-4270-9465-faea07b1363b None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:55.029439 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf4b14-c4d5-4270-9465-faea07b1363b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:55.045702 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf4b14-c4d5-4270-9465-faea07b1363b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:55.047243 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf4b14-c4d5-4270-9465-faea07b1363b None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:55.049235 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:55.065376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-90bf4b14-c4d5-4270-9465-faea07b1363b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:55.065376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:55.065376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:55.065376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:55.065376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:55.065376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:55.065376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:55.065376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:55.065376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:55.065376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:55.065376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:55.065376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:55.065376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:55.065376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:55.065376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:55.068353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:55.068353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:55.068353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:55.068353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:55.068353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:55.068353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:55.068353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:55.068353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:55.068353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:55.068353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:55.068353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:55.068353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:55.068353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:55.068353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:55.068353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:55.068353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:55.068353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:55.073673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:55.073673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:55.073673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:55.073673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:55.073673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 421/833] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:55.073673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf4b14-c4d5-4270-9465-faea07b1363b None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:55.095372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0f8c3-2c1c-431f-be09-b6dd2df5b4ae None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:55.096938 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:55.164205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ad66d-2ad5-4a62-ac1a-599a2a3d116b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:55.165276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ad66d-2ad5-4a62-ac1a-599a2a3d116b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:55.165747 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ad66d-2ad5-4a62-ac1a-599a2a3d116b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:55.166217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ad66d-2ad5-4a62-ac1a-599a2a3d116b None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:55.167658 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ad66d-2ad5-4a62-ac1a-599a2a3d116b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:55.168213 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ad66d-2ad5-4a62-ac1a-599a2a3d116b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:55.171088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ad66d-2ad5-4a62-ac1a-599a2a3d116b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:55.194865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 413/834] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:55.198984 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ad66d-2ad5-4a62-ac1a-599a2a3d116b None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:55.248400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf4b14-c4d5-4270-9465-faea07b1363b None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:55.290334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c87789-52c6-46ff-ae29-c6f899bd0933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:55.291006 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c87789-52c6-46ff-ae29-c6f899bd0933 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:55.291284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c87789-52c6-46ff-ae29-c6f899bd0933 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:55.291560 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c87789-52c6-46ff-ae29-c6f899bd0933 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:55.306220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c87789-52c6-46ff-ae29-c6f899bd0933 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:55.312211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-40c87789-52c6-46ff-ae29-c6f899bd0933 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:58:55.312211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:55.312211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:55.312211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:55.312211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:55.312211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:55.312211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:55.312211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:55.312211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:55.312211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:55.312211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:55.312211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:55.312211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:55.312211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:55.312211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:55.312211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:55.313323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:55.313323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:55.313323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:55.313323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:55.313323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:55.313323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:55.313323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:55.313323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:55.313323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:55.313323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:55.313323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:55.313323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:55.313323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:55.313323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:55.313323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:55.313323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:55.314255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:55.314255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:55.314255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:55.314255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:55.314255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:55.314255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:55.314255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:55.314255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:55.314255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:55.314255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:55.314255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:58:55.314255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:55.314255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 422/835] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:58:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:55.315003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c87789-52c6-46ff-ae29-c6f899bd0933 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:55.354457 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ad66d-2ad5-4a62-ac1a-599a2a3d116b None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:55.395387 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ec3b7-a37d-4c95-96d3-f144ce5ad5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:55.396060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ec3b7-a37d-4c95-96d3-f144ce5ad5ae None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:55.396356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ec3b7-a37d-4c95-96d3-f144ce5ad5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:55.396681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ec3b7-a37d-4c95-96d3-f144ce5ad5ae None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:55.397383 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ec3b7-a37d-4c95-96d3-f144ce5ad5ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:55.397689 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ec3b7-a37d-4c95-96d3-f144ce5ad5ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:55.399146 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ec3b7-a37d-4c95-96d3-f144ce5ad5ae None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:55.412499 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:55.420519 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 414/836] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:55.420519 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ec3b7-a37d-4c95-96d3-f144ce5ad5ae None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:55.424895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:55.439883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c87789-52c6-46ff-ae29-c6f899bd0933 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:55.484369 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9ef8a-a8a3-4bed-8f75-41aa2d1fda2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:55.485062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9ef8a-a8a3-4bed-8f75-41aa2d1fda2c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:55.485301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9ef8a-a8a3-4bed-8f75-41aa2d1fda2c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:55.485582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9ef8a-a8a3-4bed-8f75-41aa2d1fda2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:55.486644 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9ef8a-a8a3-4bed-8f75-41aa2d1fda2c None admin] RBAC: Adding query filter params (role.id=5456887a306649bf80a9ea7bd4282e97, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=3d9be74621d14c94b95ae94947e5abbd) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:55.486644 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9ef8a-a8a3-4bed-8f75-41aa2d1fda2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:55.488151 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9ef8a-a8a3-4bed-8f75-41aa2d1fda2c None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:55.496257 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 423/837] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:58:55 2022] GET /identity/v3/role_assignments?user.id=3d9be74621d14c94b95ae94947e5abbd&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:55.497337 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9ef8a-a8a3-4bed-8f75-41aa2d1fda2c None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:55.582537 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ec3b7-a37d-4c95-96d3-f144ce5ad5ae None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:55.638225 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25401925-8316-4aa0-82fe-4f8fafaa08ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:55.639497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25401925-8316-4aa0-82fe-4f8fafaa08ed None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:55.639876 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25401925-8316-4aa0-82fe-4f8fafaa08ed None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:55.640377 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25401925-8316-4aa0-82fe-4f8fafaa08ed None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:55.641378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25401925-8316-4aa0-82fe-4f8fafaa08ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:55.641811 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25401925-8316-4aa0-82fe-4f8fafaa08ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:55.644574 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25401925-8316-4aa0-82fe-4f8fafaa08ed None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:55.647345 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9ef8a-a8a3-4bed-8f75-41aa2d1fda2c None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:55.662889 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 415/838] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:58:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:55.665582 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25401925-8316-4aa0-82fe-4f8fafaa08ed None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:55.709896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4076b-c626-463b-9f69-431a0387b92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:55.710585 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4076b-c626-463b-9f69-431a0387b92a None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:55.710865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4076b-c626-463b-9f69-431a0387b92a None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:55.711330 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4076b-c626-463b-9f69-431a0387b92a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:55.721014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4076b-c626-463b-9f69-431a0387b92a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:55.722639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4076b-c626-463b-9f69-431a0387b92a None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:55.726496 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:55.739767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-8fe4076b-c626-463b-9f69-431a0387b92a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:55.739767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:55.739767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:55.739767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:55.739767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:55.739767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:55.739767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:55.739767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:55.739767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:55.739767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:55.739767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:55.739767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:55.739767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:55.739767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:55.739767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:55.740836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:55.740836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:55.740836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:55.740836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:55.740836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:55.740836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:55.740836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:55.740836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:55.740836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:55.740836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:55.740836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:55.740836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:55.740836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:55.740836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:55.740836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:55.740836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:55.740836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:55.741766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:55.741766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:55.741766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:55.741766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:55.741766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:55.741766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:55.741766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:55.741766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:55.741766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 424/839] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:55.751247 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4076b-c626-463b-9f69-431a0387b92a None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:55.758203 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:55.817833 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25401925-8316-4aa0-82fe-4f8fafaa08ed None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:55.867618 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1a7cb-255b-44f0-9511-a6a033369d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:55.868210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1a7cb-255b-44f0-9511-a6a033369d30 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:55.868569 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1a7cb-255b-44f0-9511-a6a033369d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:55.868858 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1a7cb-255b-44f0-9511-a6a033369d30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:55.885399 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b1a7cb-255b-44f0-9511-a6a033369d30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:55.886859 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4076b-c626-463b-9f69-431a0387b92a None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:55.887372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b1a7cb-255b-44f0-9511-a6a033369d30 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:55.904040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-03b1a7cb-255b-44f0-9511-a6a033369d30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:55.904040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:55.904040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:55.904040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:55.904040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:55.904040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:55.904040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:55.904040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:55.904040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:55.904040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:55.904040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:55.904040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:55.904040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:55.904040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:55.904040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:55.905069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:55.905069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:55.905069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:55.905069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:55.905069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:55.905069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:55.905069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:55.905069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:55.905069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:55.905069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:55.905069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:55.905069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:55.905069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:55.905069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:55.905069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:55.905069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:55.905069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:55.906001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:55.906001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:55.906001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:55.906001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:55.906001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 416/840] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:55.912452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1a7cb-255b-44f0-9511-a6a033369d30 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:55.939043 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b40c91-cf3b-425f-8fdf-e05de2d1e5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:55.939876 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b40c91-cf3b-425f-8fdf-e05de2d1e5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:55.940280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b40c91-cf3b-425f-8fdf-e05de2d1e5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:55.940590 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b40c91-cf3b-425f-8fdf-e05de2d1e5c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:55.941346 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b40c91-cf3b-425f-8fdf-e05de2d1e5c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:55.941855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b40c91-cf3b-425f-8fdf-e05de2d1e5c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:55.943992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b40c91-cf3b-425f-8fdf-e05de2d1e5c3 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:55.964638 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 425/841] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:58:55.969073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b40c91-cf3b-425f-8fdf-e05de2d1e5c3 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:56.036285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1a7cb-255b-44f0-9511-a6a033369d30 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:56.053208 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:56.060878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:56.092513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a884818-fd1c-4c5d-919f-ba12e601c738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:56.093148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a884818-fd1c-4c5d-919f-ba12e601c738 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:56.093433 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a884818-fd1c-4c5d-919f-ba12e601c738 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:56.093713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a884818-fd1c-4c5d-919f-ba12e601c738 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:56.094951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a884818-fd1c-4c5d-919f-ba12e601c738 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:56.095256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a884818-fd1c-4c5d-919f-ba12e601c738 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:56.096862 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a884818-fd1c-4c5d-919f-ba12e601c738 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:56.114835 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b40c91-cf3b-425f-8fdf-e05de2d1e5c3 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:56.120783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 417/842] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:56.121718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a884818-fd1c-4c5d-919f-ba12e601c738 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:56.164285 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24d773-87f1-492c-90f4-f899eee823d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:56.165356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24d773-87f1-492c-90f4-f899eee823d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:56.165869 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24d773-87f1-492c-90f4-f899eee823d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:56.166342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24d773-87f1-492c-90f4-f899eee823d5 None admin] PATH_INFO: `/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/4370f8f474c2446e989d0d8c5b75e87a/roles/6cff92a2d53f4855b38a9d9f11aa19bc` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:56.184704 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24d773-87f1-492c-90f4-f899eee823d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac9f7c74597642b0acecc956429ac4d8, user_id=4370f8f474c2446e989d0d8c5b75e87a, role_id=6cff92a2d53f4855b38a9d9f11aa19bc)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:56.186674 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24d773-87f1-492c-90f4-f899eee823d5 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:56.211347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 426/843] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:58:55 2022] PUT /identity/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/4370f8f474c2446e989d0d8c5b75e87a/roles/6cff92a2d53f4855b38a9d9f11aa19bc => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:58:56.271938 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a884818-fd1c-4c5d-919f-ba12e601c738 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:56.314244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb82a63-8944-42d4-915e-1516dcb5f025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:56.315284 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb82a63-8944-42d4-915e-1516dcb5f025 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:56.315765 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb82a63-8944-42d4-915e-1516dcb5f025 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:56.316260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb82a63-8944-42d4-915e-1516dcb5f025 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:56.326221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb82a63-8944-42d4-915e-1516dcb5f025 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:56.328044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb82a63-8944-42d4-915e-1516dcb5f025 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:56.342326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-ebb82a63-8944-42d4-915e-1516dcb5f025 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:58:56.342326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:56.342326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:56.342326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:56.342326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:56.342326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:56.342326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:56.342326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:56.342326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:56.342326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:56.342326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:56.342326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:56.342326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:58:56.342326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:58:56.342326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:58:56.342326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:58:56.343361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:56.343361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:56.343361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:58:56.343361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:56.343361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:58:56.343361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:58:56.343361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:56.343361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:56.343361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:56.343361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:56.343361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:56.343361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:56.343361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:56.343361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:56.343361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:56.343361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:56.343361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:56.343361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:56.344445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:56.344445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:56.344445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:58:56.344445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:58:56.344445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:58:56.344445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:58:56.344445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:58:56.344445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:58:56.344445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 14:58:56.344445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:56.347209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 418/844] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:58:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:56.356635 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24d773-87f1-492c-90f4-f899eee823d5 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:56.520155 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24d773-87f1-492c-90f4-f899eee823d5 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:56.532888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:56.561603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee4f7b-95f3-472f-b4e5-dc262ffc0541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:56.562335 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee4f7b-95f3-472f-b4e5-dc262ffc0541 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:56.562615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee4f7b-95f3-472f-b4e5-dc262ffc0541 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:56.562912 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee4f7b-95f3-472f-b4e5-dc262ffc0541 None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:56.563845 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee4f7b-95f3-472f-b4e5-dc262ffc0541 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:56.564210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee4f7b-95f3-472f-b4e5-dc262ffc0541 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:56.565861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee4f7b-95f3-472f-b4e5-dc262ffc0541 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:56.587562 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 427/845] 10.222.0.93 () {56 vars in 1135 bytes} [Mon Aug 1 14:58:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:56.592971 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb82a63-8944-42d4-915e-1516dcb5f025 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:56.642355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:56.716260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb82a63-8944-42d4-915e-1516dcb5f025 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:56.753566 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280262cc-3982-4704-be9d-dbbee18a30d5 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:56.753854 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280262cc-3982-4704-be9d-dbbee18a30d5 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:56.754204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280262cc-3982-4704-be9d-dbbee18a30d5 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:56.756333 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 428/846] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:56.757134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421e6ee-df08-4fd8-b7ca-56ea3d24eff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:56.758063 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421e6ee-df08-4fd8-b7ca-56ea3d24eff6 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:56.758307 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421e6ee-df08-4fd8-b7ca-56ea3d24eff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:56.758972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421e6ee-df08-4fd8-b7ca-56ea3d24eff6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:56.762331 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf05d16-f98c-4418-b038-a8b50dba11e4 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:56.762887 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf05d16-f98c-4418-b038-a8b50dba11e4 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:56.763369 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf05d16-f98c-4418-b038-a8b50dba11e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:56.785632 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-fdf05d16-f98c-4418-b038-a8b50dba11e4 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:56.787207 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-fdf05d16-f98c-4418-b038-a8b50dba11e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:56.840566 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9421e6ee-df08-4fd8-b7ca-56ea3d24eff6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:56.848512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9421e6ee-df08-4fd8-b7ca-56ea3d24eff6 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:56.886664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:56.888040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 429/847] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:58:56.912003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf05d16-f98c-4418-b038-a8b50dba11e4 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:56.961924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 419/848] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:58:56 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:58:56.963827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421e6ee-df08-4fd8-b7ca-56ea3d24eff6 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:57.068307 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:57.073897 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf05d16-f98c-4418-b038-a8b50dba11e4 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:57.126114 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1678f7c-c10f-4080-9721-b89c6d03cbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:57.126763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1678f7c-c10f-4080-9721-b89c6d03cbcc None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:57.127033 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1678f7c-c10f-4080-9721-b89c6d03cbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:57.127289 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1678f7c-c10f-4080-9721-b89c6d03cbcc None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:57.128371 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1678f7c-c10f-4080-9721-b89c6d03cbcc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:57.130210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1678f7c-c10f-4080-9721-b89c6d03cbcc None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:57.152658 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 430/849] 10.222.0.93 () {60 vars in 1141 bytes} [Mon Aug 1 14:58:56 2022] POST /identity/v3/services => generated 251 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:58:57.165810 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421e6ee-df08-4fd8-b7ca-56ea3d24eff6 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:57.221269 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f6e83-f111-411c-978e-257ebe603d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:57.222404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f6e83-f111-411c-978e-257ebe603d37 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:57.223013 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f6e83-f111-411c-978e-257ebe603d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:57.223514 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f6e83-f111-411c-978e-257ebe603d37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:57.226331 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:57.240011 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f6e83-f111-411c-978e-257ebe603d37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:57.241952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f6e83-f111-411c-978e-257ebe603d37 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:57.262131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-759f6e83-f111-411c-978e-257ebe603d37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:57.262131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:57.262131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:57.262131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:57.262131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:57.262131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:57.262131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:57.262131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:57.262131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:57.262131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:57.262131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:57.262131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:57.262131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:57.262131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:57.262131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:57.263123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:57.263123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:57.263123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:57.263123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:57.263123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:57.263123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:57.263123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:57.263123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:57.263123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:57.263123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:57.263123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:57.263123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:57.263123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:57.263123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:57.263123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:57.263123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:57.263123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:57.264083 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:57.264083 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:57.264083 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:57.264083 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:57.266644 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 420/850] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:57.272808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1678f7c-c10f-4080-9721-b89c6d03cbcc None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:57.404161 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1678f7c-c10f-4080-9721-b89c6d03cbcc None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:57.445225 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb8d58-db02-4a82-aad7-ba4902828330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:57.446398 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8d58-db02-4a82-aad7-ba4902828330 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:57.446947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8d58-db02-4a82-aad7-ba4902828330 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:57.447438 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8d58-db02-4a82-aad7-ba4902828330 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:57.448364 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb8d58-db02-4a82-aad7-ba4902828330 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:57.449001 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb8d58-db02-4a82-aad7-ba4902828330 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:57.451170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb8d58-db02-4a82-aad7-ba4902828330 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:57.466423 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 431/851] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:57.472667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f6e83-f111-411c-978e-257ebe603d37 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:57.609503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f6e83-f111-411c-978e-257ebe603d37 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:57.647774 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f7b65-b9ae-4bbe-8a0f-ee0e32b5b2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:57.648483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f7b65-b9ae-4bbe-8a0f-ee0e32b5b2af None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:57.648773 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f7b65-b9ae-4bbe-8a0f-ee0e32b5b2af None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:57.649047 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f7b65-b9ae-4bbe-8a0f-ee0e32b5b2af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:57.656941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f7b65-b9ae-4bbe-8a0f-ee0e32b5b2af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:57.658336 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f7b65-b9ae-4bbe-8a0f-ee0e32b5b2af None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:57.674309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-540f7b65-b9ae-4bbe-8a0f-ee0e32b5b2af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:57.674309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:57.674309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:57.674309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:57.674309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:57.674309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:57.674309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:57.674309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:57.674309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:57.674309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:57.674309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:57.674309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:57.674309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:58:57.674309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:58:57.674309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:58:57.675308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:58:57.675308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:57.675308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:57.675308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:57.675308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:57.675308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:57.675308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:57.675308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:57.675308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:57.675308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:57.675308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:57.675308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:57.675308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:57.675308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:57.675308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:57.675308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:57.675308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:57.677223 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:58:57.677223 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:58:57.677223 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:58:57.677223 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:58:57.677223 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:58:57.677223 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:58:57.677223 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:58:57.677223 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:57.679561 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 421/852] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:58:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:58:57.686129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb8d58-db02-4a82-aad7-ba4902828330 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:57.717540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:57.815043 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb8d58-db02-4a82-aad7-ba4902828330 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:57.855943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d811e7-5278-48fe-a141-3275225bf9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:57.856961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d811e7-5278-48fe-a141-3275225bf9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:57.857450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d811e7-5278-48fe-a141-3275225bf9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:57.857986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d811e7-5278-48fe-a141-3275225bf9a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:57.858946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d811e7-5278-48fe-a141-3275225bf9a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:57.859460 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d811e7-5278-48fe-a141-3275225bf9a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:57.861249 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d811e7-5278-48fe-a141-3275225bf9a9 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:57.876681 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 432/853] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:58:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:58:57.886200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f7b65-b9ae-4bbe-8a0f-ee0e32b5b2af None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:57.886769 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2798e-9719-421b-92b2-97c878140935 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:57.886769 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2798e-9719-421b-92b2-97c878140935 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:57.886915 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2798e-9719-421b-92b2-97c878140935 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:57.888634 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 433/854] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:57.898807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c03a72-2bd5-4dea-ade8-13d050ce4b34 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:57.899267 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c03a72-2bd5-4dea-ade8-13d050ce4b34 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:57.899734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c03a72-2bd5-4dea-ade8-13d050ce4b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:57.930444 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-79c03a72-2bd5-4dea-ade8-13d050ce4b34 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:57.932294 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-79c03a72-2bd5-4dea-ade8-13d050ce4b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:57.974190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:58.061653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 434/855] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:57 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:58:58.062862 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f7b65-b9ae-4bbe-8a0f-ee0e32b5b2af None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:58.084904 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c03a72-2bd5-4dea-ade8-13d050ce4b34 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:58.099813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:58.115253 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c5a4c-c4ac-4f66-98ce-f1b123ba5c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:58.116026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c5a4c-c4ac-4f66-98ce-f1b123ba5c67 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:58.116360 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c5a4c-c4ac-4f66-98ce-f1b123ba5c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:58.116728 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c5a4c-c4ac-4f66-98ce-f1b123ba5c67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:58.117596 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c5a4c-c4ac-4f66-98ce-f1b123ba5c67 None admin] RBAC: Adding query filter params (role.id=5456887a306649bf80a9ea7bd4282e97, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=4df37b02152c48509243b73ebbf49d8d) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:58.117995 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c5a4c-c4ac-4f66-98ce-f1b123ba5c67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:58.121016 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c5a4c-c4ac-4f66-98ce-f1b123ba5c67 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:58.130421 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 422/856] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:58:57 2022] GET /identity/v3/role_assignments?user.id=4df37b02152c48509243b73ebbf49d8d&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:58.225210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c03a72-2bd5-4dea-ade8-13d050ce4b34 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:58.269215 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57fb0f6-4c74-4d31-8360-c28d1d6d1e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:58.270961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57fb0f6-4c74-4d31-8360-c28d1d6d1e8f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:58.271574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57fb0f6-4c74-4d31-8360-c28d1d6d1e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:58.272123 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57fb0f6-4c74-4d31-8360-c28d1d6d1e8f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:58.273085 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57fb0f6-4c74-4d31-8360-c28d1d6d1e8f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:58.274771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57fb0f6-4c74-4d31-8360-c28d1d6d1e8f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:58.286528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-c57fb0f6-4c74-4d31-8360-c28d1d6d1e8f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 14:58:58.286528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:58.286528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:58:58.286528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:58:58.286528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:58:58.286528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:58:58.286528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:58:58.286528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:58:58.286528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 14:58:58.286528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:58.286528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:58:58.286528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:58.286528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:58.286528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:58.286528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:58.286528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:58.287603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:58.287603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:58.287603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:58.287603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:58.287603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:58.287603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:58.287603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:58.287603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:58:58.287603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:58:58.287603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:58:58.287603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:58:58.287603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:58.287603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:58.287603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:58.287603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:58.287603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:58.288516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:58.288516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:58.288516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:58.288516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:58.288516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:58.288516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:58.288516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:58.288516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:58.288516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:58.288516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:58.288516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:58.288516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:58:58.288516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:58:58.288516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 14:58:58.288516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:58:58.296631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 435/857] 10.222.0.93 () {56 vars in 1132 bytes} [Mon Aug 1 14:58:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:58:58.298916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c5a4c-c4ac-4f66-98ce-f1b123ba5c67 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:58.432949 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c5a4c-c4ac-4f66-98ce-f1b123ba5c67 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:58.447583 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:58.491602 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f221857-adcc-4fd1-a56f-ff0d279b16ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:58.492522 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f221857-adcc-4fd1-a56f-ff0d279b16ec None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:58.492816 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f221857-adcc-4fd1-a56f-ff0d279b16ec None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:58.493097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f221857-adcc-4fd1-a56f-ff0d279b16ec None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:58.493986 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f221857-adcc-4fd1-a56f-ff0d279b16ec None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:58.494516 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f221857-adcc-4fd1-a56f-ff0d279b16ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:58.497652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f221857-adcc-4fd1-a56f-ff0d279b16ec None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:58.511285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 423/858] 10.222.0.93 () {56 vars in 1126 bytes} [Mon Aug 1 14:58:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:58.519187 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57fb0f6-4c74-4d31-8360-c28d1d6d1e8f None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:58.669068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:58.728374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57fb0f6-4c74-4d31-8360-c28d1d6d1e8f None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:58.769249 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d209b-9032-43d6-993c-217b5ce74b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:58.769856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d209b-9032-43d6-993c-217b5ce74b9d None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:58.770139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d209b-9032-43d6-993c-217b5ce74b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:58.770509 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d209b-9032-43d6-993c-217b5ce74b9d None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:58.771282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d209b-9032-43d6-993c-217b5ce74b9d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:58.771766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d209b-9032-43d6-993c-217b5ce74b9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:58.773380 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d209b-9032-43d6-993c-217b5ce74b9d None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:58.784643 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 436/859] 10.222.0.93 () {56 vars in 1126 bytes} [Mon Aug 1 14:58:58 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:58.813503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818170d-870d-49ec-9d42-222952d14310 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:58.813843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818170d-870d-49ec-9d42-222952d14310 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:58.814054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818170d-870d-49ec-9d42-222952d14310 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:58.816017 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 424/860] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:58.822754 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa7563-c911-4b66-b4d4-e83e9b82bd71 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:58.823102 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa7563-c911-4b66-b4d4-e83e9b82bd71 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:58.823450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa7563-c911-4b66-b4d4-e83e9b82bd71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:58.848767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-beaa7563-c911-4b66-b4d4-e83e9b82bd71 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:58.850290 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-beaa7563-c911-4b66-b4d4-e83e9b82bd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:58.945714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 437/861] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:58 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:58:58.970639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818170d-870d-49ec-9d42-222952d14310 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:59.124104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818170d-870d-49ec-9d42-222952d14310 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:59.182932 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabf69d-3c8e-49ef-8eb5-cbc192bc256f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:59.183603 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabf69d-3c8e-49ef-8eb5-cbc192bc256f None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:59.183885 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabf69d-3c8e-49ef-8eb5-cbc192bc256f None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:59.184159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabf69d-3c8e-49ef-8eb5-cbc192bc256f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:59.198390 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabf69d-3c8e-49ef-8eb5-cbc192bc256f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:58:59.203664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-2dabf69d-3c8e-49ef-8eb5-cbc192bc256f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:59.203664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:59.203664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:59.203664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:59.203664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:59.203664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:59.203664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:59.203664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:59.203664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:59.203664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:59.203664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:59.203664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:59.203664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:58:59.203664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:58:59.203664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:58:59.203664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:58:59.204736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:58:59.204736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:58:59.204736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:58:59.204736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:58:59.204736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:59.204736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:59.204736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:59.204736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:59.204736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:59.204736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:59.204736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:59.204736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:59.204736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:59.204736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:59.204736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:59.204736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:59.205591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:59.205591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:59.205591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:59.205591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:59.205591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:58:59.205591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:58:59.205591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:58:59.205591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:58:59.205591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:58:59.205591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:58:59.205591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 14:58:59.205591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:59.205591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 425/862] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 14:58:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:59.214612 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa7563-c911-4b66-b4d4-e83e9b82bd71 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:59.299631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:59.341665 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa7563-c911-4b66-b4d4-e83e9b82bd71 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:59.398091 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623ecb4-a7a7-4bfe-9c48-5095860ab487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:59.398679 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623ecb4-a7a7-4bfe-9c48-5095860ab487 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:59.399069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623ecb4-a7a7-4bfe-9c48-5095860ab487 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:59.399479 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623ecb4-a7a7-4bfe-9c48-5095860ab487 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:59.400216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623ecb4-a7a7-4bfe-9c48-5095860ab487 None admin] RBAC: Adding query filter params (name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:59.400564 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623ecb4-a7a7-4bfe-9c48-5095860ab487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:59.402211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623ecb4-a7a7-4bfe-9c48-5095860ab487 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:59.412937 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 438/863] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 14:58:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:59.419289 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabf69d-3c8e-49ef-8eb5-cbc192bc256f None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:59.472697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:59.555094 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabf69d-3c8e-49ef-8eb5-cbc192bc256f None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:59.592632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a6749-0d48-4fa5-a543-5008efaa4611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:59.593372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a6749-0d48-4fa5-a543-5008efaa4611 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:59.593652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a6749-0d48-4fa5-a543-5008efaa4611 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:59.593924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a6749-0d48-4fa5-a543-5008efaa4611 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:59.605700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a6749-0d48-4fa5-a543-5008efaa4611 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:59.607200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a6749-0d48-4fa5-a543-5008efaa4611 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:59.624503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-7d4a6749-0d48-4fa5-a543-5008efaa4611 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:59.624503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:58:59.624503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:58:59.624503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:58:59.624503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:58:59.624503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:58:59.624503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:58:59.624503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:58:59.624503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:58:59.624503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:58:59.624503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:58:59.624503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:58:59.624503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:58:59.624503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:58:59.624503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:58:59.626724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:58:59.626724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:58:59.626724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:58:59.626724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:58:59.626724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:58:59.626724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:58:59.626724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:58:59.626724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:58:59.626724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:58:59.626724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:58:59.626724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:58:59.626724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:58:59.626724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:58:59.626724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:58:59.626724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:58:59.626724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:58:59.626724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:58:59.629012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:58:59.629012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:58:59.629012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:58:59.629012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:58:59.629012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 426/864] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:58:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:58:59.632196 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623ecb4-a7a7-4bfe-9c48-5095860ab487 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:59.649199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159fb47-f223-42dd-a97d-738cf5c0ec02 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:59.649490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159fb47-f223-42dd-a97d-738cf5c0ec02 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:59.649646 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159fb47-f223-42dd-a97d-738cf5c0ec02 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:59.651692 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 427/865] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:58:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:58:59.660018 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d72477-cb23-401a-b97b-05d115159cdb None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:59.661667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d72477-cb23-401a-b97b-05d115159cdb None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:59.661946 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d72477-cb23-401a-b97b-05d115159cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:59.688952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-47d72477-cb23-401a-b97b-05d115159cdb None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:58:59.690192 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-47d72477-cb23-401a-b97b-05d115159cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:58:59.792595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 428/866] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:58:59 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:58:59.811700 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623ecb4-a7a7-4bfe-9c48-5095860ab487 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:58:59.817439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d72477-cb23-401a-b97b-05d115159cdb None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:59.856921 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:59.857427 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:58:59.874454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957fd116-e887-407e-bba4-af7134dc813e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:58:59.876531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957fd116-e887-407e-bba4-af7134dc813e None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:58:59.876531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957fd116-e887-407e-bba4-af7134dc813e None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:58:59.876531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957fd116-e887-407e-bba4-af7134dc813e None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:58:59.878194 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957fd116-e887-407e-bba4-af7134dc813e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:58:59.878786 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957fd116-e887-407e-bba4-af7134dc813e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:58:59.882076 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957fd116-e887-407e-bba4-af7134dc813e None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:58:59.906853 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 439/867] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:58:59.908557 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957fd116-e887-407e-bba4-af7134dc813e None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:58:59.982963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d72477-cb23-401a-b97b-05d115159cdb None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:00.040201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2a243-bf99-42d1-9411-352488dd9023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:00.041279 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2a243-bf99-42d1-9411-352488dd9023 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:00.041833 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2a243-bf99-42d1-9411-352488dd9023 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:00.042449 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2a243-bf99-42d1-9411-352488dd9023 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:00.043345 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2a243-bf99-42d1-9411-352488dd9023 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:00.045074 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2a243-bf99-42d1-9411-352488dd9023 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:00.059446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-c3e2a243-bf99-42d1-9411-352488dd9023 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 14:59:00.059446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:00.059446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:59:00.059446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:59:00.059446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:59:00.059446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:59:00.059446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:59:00.059446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:00.059446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 14:59:00.059446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:00.059446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:59:00.059446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:00.059446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:00.059446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:00.059446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:00.059446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:00.059446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:00.060611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:00.060611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:00.060611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:00.060611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:00.060611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:00.060611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:00.060611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:59:00.060611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:59:00.060611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:59:00.060611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:59:00.060611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:00.060611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:00.060611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:00.060611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:00.060611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:00.060611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:00.061488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:00.061488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:00.061488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:00.061488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:00.061488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:00.061488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:00.061488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:00.061488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:00.061488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:00.061488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:00.061488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:59:00.061488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:00.061488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 14:59:00.061488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:00.064552 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 429/868] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:58:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:00.071006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2a243-bf99-42d1-9411-352488dd9023 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:00.091095 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957fd116-e887-407e-bba4-af7134dc813e None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:00.144685 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433220b-8e1d-485a-ac1b-6b738e82eb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:00.146404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433220b-8e1d-485a-ac1b-6b738e82eb40 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:00.146836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433220b-8e1d-485a-ac1b-6b738e82eb40 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:00.147294 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433220b-8e1d-485a-ac1b-6b738e82eb40 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:00.158438 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433220b-8e1d-485a-ac1b-6b738e82eb40 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:00.160799 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433220b-8e1d-485a-ac1b-6b738e82eb40 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:00.181301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-2433220b-8e1d-485a-ac1b-6b738e82eb40 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:59:00.181301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:00.181301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:00.181301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:00.181301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:00.181301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:00.181301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:00.181301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:00.181301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:00.181301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:00.181301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:00.181301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:00.181301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:59:00.181301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:59:00.181301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:59:00.181301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:59:00.182533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:00.182533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:00.182533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:59:00.182533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:59:00.182533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:59:00.182533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:59:00.182533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:00.182533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:00.182533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:00.182533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:00.182533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:00.182533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:00.182533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:00.182533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:00.182533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:00.182533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:00.182533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:00.182533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:00.183726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:00.183726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:00.183726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:59:00.183726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:59:00.183726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:59:00.183726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:59:00.183726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:59:00.183726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:59:00.183726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:59:00.183726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:00.187019 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 440/869] 10.222.0.93 () {56 vars in 1117 bytes} [Mon Aug 1 14:58:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:00.195115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433220b-8e1d-485a-ac1b-6b738e82eb40 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:00.199331 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:00.226696 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2a243-bf99-42d1-9411-352488dd9023 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:00.274038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:00.275914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6b2ca-273c-4e44-8e38-adbcb196653a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:00.277166 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6b2ca-273c-4e44-8e38-adbcb196653a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:00.278301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6b2ca-273c-4e44-8e38-adbcb196653a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:00.278590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6b2ca-273c-4e44-8e38-adbcb196653a None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:00.279300 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6b2ca-273c-4e44-8e38-adbcb196653a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:00.279615 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6b2ca-273c-4e44-8e38-adbcb196653a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:00.281071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6b2ca-273c-4e44-8e38-adbcb196653a None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:00.292370 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 430/870] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:59:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:00.293749 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6b2ca-273c-4e44-8e38-adbcb196653a None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:00.352356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433220b-8e1d-485a-ac1b-6b738e82eb40 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:00.415117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c766dff-1a5c-43ce-b468-5f1ba75f01df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:00.416078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c766dff-1a5c-43ce-b468-5f1ba75f01df None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:00.416606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c766dff-1a5c-43ce-b468-5f1ba75f01df None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:00.417670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c766dff-1a5c-43ce-b468-5f1ba75f01df None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:00.418996 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c766dff-1a5c-43ce-b468-5f1ba75f01df None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:00.419522 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c766dff-1a5c-43ce-b468-5f1ba75f01df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:00.422149 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c766dff-1a5c-43ce-b468-5f1ba75f01df None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:00.429923 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6b2ca-273c-4e44-8e38-adbcb196653a None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:00.445971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 441/871] 10.222.0.93 () {56 vars in 1149 bytes} [Mon Aug 1 14:59:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:00.447280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6fa45-ce78-4420-8b5a-36c42460e82a None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:00.447687 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6fa45-ce78-4420-8b5a-36c42460e82a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:00.448783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6fa45-ce78-4420-8b5a-36c42460e82a None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:00.449417 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 442/872] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:00.453326 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6fa45-ce78-4420-8b5a-36c42460e82a None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:00.483484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102c69d-6517-4655-a25b-ae8646aa84db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:00.484130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102c69d-6517-4655-a25b-ae8646aa84db None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:00.484411 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102c69d-6517-4655-a25b-ae8646aa84db None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:00.484717 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102c69d-6517-4655-a25b-ae8646aa84db None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:00.485583 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102c69d-6517-4655-a25b-ae8646aa84db None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:00.485692 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102c69d-6517-4655-a25b-ae8646aa84db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:00.487076 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102c69d-6517-4655-a25b-ae8646aa84db None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:00.492700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 431/873] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:59:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:00.493847 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbbf6a-0a8a-4370-bf1c-96a87af4812c None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:00.494277 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbbf6a-0a8a-4370-bf1c-96a87af4812c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:00.494480 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbbf6a-0a8a-4370-bf1c-96a87af4812c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:00.513788 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-5bfbbf6a-0a8a-4370-bf1c-96a87af4812c None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:00.515007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-5bfbbf6a-0a8a-4370-bf1c-96a87af4812c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:00.517315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:00.608221 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6fa45-ce78-4420-8b5a-36c42460e82a None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:00.618952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:00.640535 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 432/874] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:00 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:00.640535 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbbf6a-0a8a-4370-bf1c-96a87af4812c None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:00.669151 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323149c-1af3-4502-90c6-da2239f7cd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:00.669702 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323149c-1af3-4502-90c6-da2239f7cd3d None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:00.669980 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323149c-1af3-4502-90c6-da2239f7cd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:00.670257 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323149c-1af3-4502-90c6-da2239f7cd3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:00.759309 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323149c-1af3-4502-90c6-da2239f7cd3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:00.760804 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323149c-1af3-4502-90c6-da2239f7cd3d None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:00.765016 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbbf6a-0a8a-4370-bf1c-96a87af4812c None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:00.819812 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8001747-bedc-420a-a909-d9fb6e24627b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:00.820643 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8001747-bedc-420a-a909-d9fb6e24627b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:00.820948 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8001747-bedc-420a-a909-d9fb6e24627b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:00.821272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8001747-bedc-420a-a909-d9fb6e24627b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:00.822039 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8001747-bedc-420a-a909-d9fb6e24627b None admin] RBAC: Adding query filter params (interface=public, service_id=513defa487124610aff4a2a8a38c36b7, region_id=RegionOne) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:00.822353 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8001747-bedc-420a-a909-d9fb6e24627b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:00.823765 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8001747-bedc-420a-a909-d9fb6e24627b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:00.833337 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 433/875] 10.222.0.93 () {56 vars in 1251 bytes} [Mon Aug 1 14:59:00 2022] GET /identity/v3/endpoints?service_id=513defa487124610aff4a2a8a38c36b7&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:00.834145 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8001747-bedc-420a-a909-d9fb6e24627b None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:00.838570 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:00.905504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 443/876] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:59:00 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 14:59:00.906384 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323149c-1af3-4502-90c6-da2239f7cd3d None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:00.917497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:00.967442 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8001747-bedc-420a-a909-d9fb6e24627b None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:01.012955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812bb17-c4e2-4eef-9ec1-11e633bd6530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:01.014100 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812bb17-c4e2-4eef-9ec1-11e633bd6530 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:01.014441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812bb17-c4e2-4eef-9ec1-11e633bd6530 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:01.014881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812bb17-c4e2-4eef-9ec1-11e633bd6530 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:01.015899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812bb17-c4e2-4eef-9ec1-11e633bd6530 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:01.018109 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812bb17-c4e2-4eef-9ec1-11e633bd6530 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:01.035972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-a812bb17-c4e2-4eef-9ec1-11e633bd6530 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:59:01.035972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:01.035972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:59:01.035972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:59:01.035972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:59:01.035972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:59:01.035972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:59:01.035972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:01.035972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:59:01.035972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:01.035972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:59:01.035972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:01.035972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:01.035972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:01.035972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:01.035972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:01.035972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:01.038211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:01.038211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:01.038211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:01.038211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:01.038211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:01.038211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:01.038211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:59:01.038211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:59:01.038211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:59:01.038211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:59:01.038211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:01.038211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:01.038211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:01.038211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:01.038211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:01.038211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:01.039931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:01.039931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:01.039931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:01.039931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:01.039931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:01.039931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:01.039931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:01.039931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:01.039931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:01.039931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:01.039931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:59:01.039931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:01.039931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:59:01.039931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:01.039931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 434/877] 10.222.0.93 () {56 vars in 1126 bytes} [Mon Aug 1 14:59:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:01.041091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812bb17-c4e2-4eef-9ec1-11e633bd6530 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:01.052787 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323149c-1af3-4502-90c6-da2239f7cd3d None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:01.100034 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b19f24-5586-4ffb-9d34-b688ac587cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:01.100618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b19f24-5586-4ffb-9d34-b688ac587cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:01.100898 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b19f24-5586-4ffb-9d34-b688ac587cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:01.101179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b19f24-5586-4ffb-9d34-b688ac587cf3 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:01.101834 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b19f24-5586-4ffb-9d34-b688ac587cf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:01.103217 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b19f24-5586-4ffb-9d34-b688ac587cf3 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:01.108975 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 444/878] 10.222.0.93 () {56 vars in 1087 bytes} [Mon Aug 1 14:59:00 2022] GET /identity/v3/services => generated 1011 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:59:01.109789 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b19f24-5586-4ffb-9d34-b688ac587cf3 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:01.139768 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:01.185183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812bb17-c4e2-4eef-9ec1-11e633bd6530 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:01.218229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:01.230456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f98c92-482a-445f-a333-defe183bf327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:01.231786 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f98c92-482a-445f-a333-defe183bf327 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:01.232073 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f98c92-482a-445f-a333-defe183bf327 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:01.232350 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f98c92-482a-445f-a333-defe183bf327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:01.245775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f98c92-482a-445f-a333-defe183bf327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:01.247272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f98c92-482a-445f-a333-defe183bf327 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:01.264042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-97f98c92-482a-445f-a333-defe183bf327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:01.264042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:01.264042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:01.264042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:01.264042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:01.264042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:01.264042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:01.264042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:01.264042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:01.264042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:01.264042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:01.264042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:01.264042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:59:01.264042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:59:01.264042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:59:01.265181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:59:01.265181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:01.265181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:01.265181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:01.265181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:01.265181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:01.265181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:01.265181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:01.265181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:01.265181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:01.265181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:01.265181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:01.265181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:01.265181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:01.265181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:01.265181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:01.265181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:01.266109 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:59:01.266109 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:59:01.266109 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:01.266109 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:01.266109 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 435/879] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:01.267148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fc3f2-baac-4bf3-836f-3ee5c27c380f None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:01.267431 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fc3f2-baac-4bf3-836f-3ee5c27c380f None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:01.267707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fc3f2-baac-4bf3-836f-3ee5c27c380f None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:01.269298 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 436/880] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:01.276425 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fc3f2-baac-4bf3-836f-3ee5c27c380f None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:01.279685 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b19f24-5586-4ffb-9d34-b688ac587cf3 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:01.328030 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69292883-c52f-40b5-8eee-ee28b7c645f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:01.328703 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69292883-c52f-40b5-8eee-ee28b7c645f2 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:01.328990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69292883-c52f-40b5-8eee-ee28b7c645f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:01.329262 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69292883-c52f-40b5-8eee-ee28b7c645f2 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:01.329945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69292883-c52f-40b5-8eee-ee28b7c645f2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:01.330266 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69292883-c52f-40b5-8eee-ee28b7c645f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:01.331656 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69292883-c52f-40b5-8eee-ee28b7c645f2 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:01.339701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 445/881] 10.222.0.93 () {56 vars in 1122 bytes} [Mon Aug 1 14:59:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:01.340906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4da79-8f9d-4c1e-a45b-b4a570baf526 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:01.341191 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4da79-8f9d-4c1e-a45b-b4a570baf526 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:01.341466 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4da79-8f9d-4c1e-a45b-b4a570baf526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:01.362174 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-a8d4da79-8f9d-4c1e-a45b-b4a570baf526 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:01.363406 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a8d4da79-8f9d-4c1e-a45b-b4a570baf526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:01.409466 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fc3f2-baac-4bf3-836f-3ee5c27c380f None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:01.431802 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:01.443559 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 446/882] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:01 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:01.445656 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4da79-8f9d-4c1e-a45b-b4a570baf526 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:01.449506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79f988-9294-4085-82ce-7fd83d23463a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:01.450580 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79f988-9294-4085-82ce-7fd83d23463a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:01.450580 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79f988-9294-4085-82ce-7fd83d23463a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:01.451040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79f988-9294-4085-82ce-7fd83d23463a None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:01.451899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79f988-9294-4085-82ce-7fd83d23463a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:01.452294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79f988-9294-4085-82ce-7fd83d23463a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:01.453860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79f988-9294-4085-82ce-7fd83d23463a None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:01.467985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 437/883] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:01.469130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79f988-9294-4085-82ce-7fd83d23463a None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:01.527691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:01.572000 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4da79-8f9d-4c1e-a45b-b4a570baf526 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:01.587546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79f988-9294-4085-82ce-7fd83d23463a None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:01.614783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a5ca9-7928-48b5-a495-073d7387fb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:01.615505 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a5ca9-7928-48b5-a495-073d7387fb53 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:01.615855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a5ca9-7928-48b5-a495-073d7387fb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:01.616201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a5ca9-7928-48b5-a495-073d7387fb53 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:01.616991 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a5ca9-7928-48b5-a495-073d7387fb53 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:01.617371 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a5ca9-7928-48b5-a495-073d7387fb53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:01.618858 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a5ca9-7928-48b5-a495-073d7387fb53 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:01.624499 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 447/884] 10.222.0.93 () {56 vars in 1122 bytes} [Mon Aug 1 14:59:01 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:01.628816 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990155fd-8607-401f-9c0d-e64e770d3b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:01.629980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990155fd-8607-401f-9c0d-e64e770d3b16 None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:01.630258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990155fd-8607-401f-9c0d-e64e770d3b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:01.630525 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990155fd-8607-401f-9c0d-e64e770d3b16 None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:01.631240 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990155fd-8607-401f-9c0d-e64e770d3b16 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:01.632715 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990155fd-8607-401f-9c0d-e64e770d3b16 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:01.634687 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a5ca9-7928-48b5-a495-073d7387fb53 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:01.655788 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 438/885] 10.222.0.93 () {60 vars in 1141 bytes} [Mon Aug 1 14:59:01 2022] POST /identity/v3/services => generated 269 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:59:01.816511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a5ca9-7928-48b5-a495-073d7387fb53 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:01.868214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:01.890547 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e00d18-b8d7-4d8f-8b2b-f82e65221268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:01.891267 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e00d18-b8d7-4d8f-8b2b-f82e65221268 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:01.891626 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e00d18-b8d7-4d8f-8b2b-f82e65221268 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:01.891908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e00d18-b8d7-4d8f-8b2b-f82e65221268 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:01.901386 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e00d18-b8d7-4d8f-8b2b-f82e65221268 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:01.902971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e00d18-b8d7-4d8f-8b2b-f82e65221268 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:01.916838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-04e00d18-b8d7-4d8f-8b2b-f82e65221268 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:59:01.916838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:01.916838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:01.916838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:01.916838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:01.916838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:01.916838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:01.916838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:01.916838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:01.916838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:01.916838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:01.916838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:01.916838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:59:01.916838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:59:01.916838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:59:01.918100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:59:01.918100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:01.918100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:01.918100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:01.918100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:01.918100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:01.918100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:01.918100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:01.918100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:01.918100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:01.918100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:01.918100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:01.918100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:01.918100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:01.918100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:01.918100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:01.918100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:01.919512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:59:01.919512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:59:01.919512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:59:01.919512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:59:01.919512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:59:01.919512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:59:01.919512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:59:01.919512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:01.919512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 448/886] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:59:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:01.925675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990155fd-8607-401f-9c0d-e64e770d3b16 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:01.976376 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:02.060302 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990155fd-8607-401f-9c0d-e64e770d3b16 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:02.102128 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0bc7c-4955-427a-8e05-1dd76204c126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:02.102820 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0bc7c-4955-427a-8e05-1dd76204c126 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:02.103289 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0bc7c-4955-427a-8e05-1dd76204c126 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:02.103613 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0bc7c-4955-427a-8e05-1dd76204c126 None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:02.104375 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0bc7c-4955-427a-8e05-1dd76204c126 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:02.104709 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0bc7c-4955-427a-8e05-1dd76204c126 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:02.106252 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0bc7c-4955-427a-8e05-1dd76204c126 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:02.121366 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 439/887] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:59:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:02.127429 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e00d18-b8d7-4d8f-8b2b-f82e65221268 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:02.258887 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e00d18-b8d7-4d8f-8b2b-f82e65221268 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:02.301996 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e25ba-d68d-4dd4-8f7d-0d86087be3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:02.303007 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e25ba-d68d-4dd4-8f7d-0d86087be3bd None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:02.303653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e25ba-d68d-4dd4-8f7d-0d86087be3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:02.304166 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e25ba-d68d-4dd4-8f7d-0d86087be3bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:02.305255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e25ba-d68d-4dd4-8f7d-0d86087be3bd None admin] RBAC: Adding query filter params (role.id=6cff92a2d53f4855b38a9d9f11aa19bc, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=4370f8f474c2446e989d0d8c5b75e87a) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:02.305785 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e25ba-d68d-4dd4-8f7d-0d86087be3bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:02.307441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e25ba-d68d-4dd4-8f7d-0d86087be3bd None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:02.315298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 449/888] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:59:02 2022] GET /identity/v3/role_assignments?user.id=4370f8f474c2446e989d0d8c5b75e87a&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=6cff92a2d53f4855b38a9d9f11aa19bc => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:03.435837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a91755-c414-42b0-90fe-c859e6c7efe2 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:03.436374 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a91755-c414-42b0-90fe-c859e6c7efe2 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:03.436374 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a91755-c414-42b0-90fe-c859e6c7efe2 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:03.438077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 440/889] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:03.444163 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83805a-6ae1-436e-b163-269f5406367c None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:03.444677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83805a-6ae1-436e-b163-269f5406367c None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:03.444757 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83805a-6ae1-436e-b163-269f5406367c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:03.465335 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-0f83805a-6ae1-436e-b163-269f5406367c None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:03.466787 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0f83805a-6ae1-436e-b163-269f5406367c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:03.537461 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:03.587946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 450/890] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:03.630543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a91755-c414-42b0-90fe-c859e6c7efe2 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:03.757811 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:03.763755 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a91755-c414-42b0-90fe-c859e6c7efe2 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:03.813382 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2528ee-3be0-4e5e-8640-68a069dcb505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:03.814053 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2528ee-3be0-4e5e-8640-68a069dcb505 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:03.814345 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2528ee-3be0-4e5e-8640-68a069dcb505 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:03.814613 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2528ee-3be0-4e5e-8640-68a069dcb505 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:03.815244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2528ee-3be0-4e5e-8640-68a069dcb505 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:03.817279 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2528ee-3be0-4e5e-8640-68a069dcb505 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:03.828186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-db2528ee-3be0-4e5e-8640-68a069dcb505 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 14:59:03.828186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:03.828186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:59:03.828186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:59:03.828186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:59:03.828186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:59:03.828186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:59:03.828186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:03.828186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 14:59:03.828186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:03.828186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:59:03.828186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:03.828186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:03.828186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:03.828186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:03.828186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:03.828186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:03.830190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:03.830190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:03.830190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:03.830190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:03.830190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:03.830190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:03.830190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:59:03.830190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:59:03.830190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:59:03.830190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:59:03.830190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:03.830190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:03.830190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:03.830190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:03.830190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:03.830190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:03.834188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:03.834188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:03.834188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:03.834188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:03.834188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:03.834188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:03.834188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:03.834188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:03.834188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:03.834188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:03.834188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:59:03.834188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:03.834188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 14:59:03.834188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:03.834188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 441/891] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:59:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:03.837193 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83805a-6ae1-436e-b163-269f5406367c None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:03.980404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9850bcb-03a4-48b7-b5af-15d2d6bfd133 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:03.980804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9850bcb-03a4-48b7-b5af-15d2d6bfd133 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:03.981011 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9850bcb-03a4-48b7-b5af-15d2d6bfd133 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:03.982747 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 442/892] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:03.984252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83805a-6ae1-436e-b163-269f5406367c None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:03.996420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ad2a3-dfcc-4356-a026-881b865df4cb None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:03.997072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ad2a3-dfcc-4356-a026-881b865df4cb None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:03.997577 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ad2a3-dfcc-4356-a026-881b865df4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:04.018937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-aa5ad2a3-dfcc-4356-a026-881b865df4cb None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:04.020549 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-aa5ad2a3-dfcc-4356-a026-881b865df4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:04.028400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58bd5a-9b97-4836-9401-4f5f72b1d634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:04.029074 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58bd5a-9b97-4836-9401-4f5f72b1d634 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:04.029358 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58bd5a-9b97-4836-9401-4f5f72b1d634 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:04.029640 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58bd5a-9b97-4836-9401-4f5f72b1d634 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:04.030336 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb58bd5a-9b97-4836-9401-4f5f72b1d634 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:04.030645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb58bd5a-9b97-4836-9401-4f5f72b1d634 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:04.032074 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb58bd5a-9b97-4836-9401-4f5f72b1d634 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:04.038164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 451/893] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:59:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:04.043805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58bd5a-9b97-4836-9401-4f5f72b1d634 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:04.117185 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:04.143446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 443/894] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:59:04.145485 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43df3a-54aa-45ac-ae5b-5eb124ee8fd8 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:04.145886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43df3a-54aa-45ac-ae5b-5eb124ee8fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:04.146337 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43df3a-54aa-45ac-ae5b-5eb124ee8fd8 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:04.148247 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 444/895] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:04.155220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da8da0-b938-4780-af3f-0d097a1c0325 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:04.155685 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da8da0-b938-4780-af3f-0d097a1c0325 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:04.156942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da8da0-b938-4780-af3f-0d097a1c0325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:04.182226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-f7da8da0-b938-4780-af3f-0d097a1c0325 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:04.183479 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-f7da8da0-b938-4780-af3f-0d097a1c0325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:04.196047 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58bd5a-9b97-4836-9401-4f5f72b1d634 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:04.239116 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d54469-7957-4fa3-a054-6783f46eb799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:04.239740 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d54469-7957-4fa3-a054-6783f46eb799 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:04.240017 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d54469-7957-4fa3-a054-6783f46eb799 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:04.240291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d54469-7957-4fa3-a054-6783f46eb799 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:04.241002 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d54469-7957-4fa3-a054-6783f46eb799 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:04.241308 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d54469-7957-4fa3-a054-6783f46eb799 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:04.242711 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d54469-7957-4fa3-a054-6783f46eb799 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:04.253735 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 452/896] 10.222.0.93 () {56 vars in 1114 bytes} [Mon Aug 1 14:59:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:04.255272 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d54469-7957-4fa3-a054-6783f46eb799 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:04.266969 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 445/897] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:59:04.268107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da8da0-b938-4780-af3f-0d097a1c0325 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:04.304418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:04.419199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da8da0-b938-4780-af3f-0d097a1c0325 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:04.426665 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d54469-7957-4fa3-a054-6783f46eb799 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:04.443687 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:04.470799 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c58e5d-6c77-4122-94f3-5fbfad5d90d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:04.471479 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c58e5d-6c77-4122-94f3-5fbfad5d90d6 None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:04.471896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c58e5d-6c77-4122-94f3-5fbfad5d90d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:04.472176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c58e5d-6c77-4122-94f3-5fbfad5d90d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:04.473863 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c58e5d-6c77-4122-94f3-5fbfad5d90d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:04.475355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c58e5d-6c77-4122-94f3-5fbfad5d90d6 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:04.490336 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92491622-2d04-4aa0-9079-16712f77b1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:04.492077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92491622-2d04-4aa0-9079-16712f77b1ff None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:04.492778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92491622-2d04-4aa0-9079-16712f77b1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:04.493239 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92491622-2d04-4aa0-9079-16712f77b1ff None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:04.494372 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92491622-2d04-4aa0-9079-16712f77b1ff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:04.496689 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92491622-2d04-4aa0-9079-16712f77b1ff None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:04.506243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 446/898] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 14:59:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:59:04.506243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c58e5d-6c77-4122-94f3-5fbfad5d90d6 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:04.545786 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 453/899] 10.222.0.93 () {60 vars in 1141 bytes} [Mon Aug 1 14:59:04 2022] POST /identity/v3/services => generated 241 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:59:04.629932 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:04.676632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c58e5d-6c77-4122-94f3-5fbfad5d90d6 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:04.727924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d7e6e-e595-4144-872a-5f74c68512ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:04.728625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d7e6e-e595-4144-872a-5f74c68512ce None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:04.728910 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d7e6e-e595-4144-872a-5f74c68512ce None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:04.729184 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d7e6e-e595-4144-872a-5f74c68512ce None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:04.729885 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761d7e6e-e595-4144-872a-5f74c68512ce None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:04.731265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761d7e6e-e595-4144-872a-5f74c68512ce None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:04.742170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-761d7e6e-e595-4144-872a-5f74c68512ce None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 14:59:04.742170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:04.742170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:59:04.742170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:59:04.742170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:59:04.742170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:59:04.742170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:59:04.742170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:04.742170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 14:59:04.742170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:04.742170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:59:04.742170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:04.742170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:04.742170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:04.742170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:04.742170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:04.743234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:04.743234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:04.743234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:04.743234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:04.743234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:04.743234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:04.743234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:04.743234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:59:04.743234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:59:04.743234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:59:04.743234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:59:04.743234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:04.743234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:04.743234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:04.743234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:04.743234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:04.744199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:04.744199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:04.744199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:04.744199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:04.744199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:04.744199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:04.744199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:04.744199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:04.744199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:04.744199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:04.744199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:04.744199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:59:04.744199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:04.744199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 14:59:04.744199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:04.747067 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9baf0a-4258-494a-8ea1-d5032d45be87 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:04.747426 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9baf0a-4258-494a-8ea1-d5032d45be87 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:04.747572 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 447/900] 10.222.0.93 () {56 vars in 1132 bytes} [Mon Aug 1 14:59:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:04.747714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9baf0a-4258-494a-8ea1-d5032d45be87 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:04.749924 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 454/901] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:04.750596 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d7e6e-e595-4144-872a-5f74c68512ce None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:04.755150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3925c8-5521-41f1-a861-3878cc53cdc6 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:04.756376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3925c8-5521-41f1-a861-3878cc53cdc6 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:04.756376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3925c8-5521-41f1-a861-3878cc53cdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:04.782174 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-8e3925c8-5521-41f1-a861-3878cc53cdc6 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:04.783408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-8e3925c8-5521-41f1-a861-3878cc53cdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:04.881755 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d7e6e-e595-4144-872a-5f74c68512ce None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:04.895403 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 455/902] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:59:04.919186 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3925c8-5521-41f1-a861-3878cc53cdc6 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:04.924874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff6b4e-96d2-468f-9141-e0e5b499f8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:04.925698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff6b4e-96d2-468f-9141-e0e5b499f8ef None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:04.926051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff6b4e-96d2-468f-9141-e0e5b499f8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:04.926400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff6b4e-96d2-468f-9141-e0e5b499f8ef None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:04.927232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff6b4e-96d2-468f-9141-e0e5b499f8ef None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:04.927911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff6b4e-96d2-468f-9141-e0e5b499f8ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:04.929222 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff6b4e-96d2-468f-9141-e0e5b499f8ef None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:04.935798 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 448/903] 10.222.0.93 () {56 vars in 1126 bytes} [Mon Aug 1 14:59:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:04.936871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:04.941774 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff6b4e-96d2-468f-9141-e0e5b499f8ef None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:05.050758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:05.061124 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3925c8-5521-41f1-a861-3878cc53cdc6 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:05.071908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff6b4e-96d2-468f-9141-e0e5b499f8ef None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:05.103727 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8454c78-9b0f-4446-ad0b-ffaf2a35ab5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:05.104766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8454c78-9b0f-4446-ad0b-ffaf2a35ab5d None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:05.105244 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8454c78-9b0f-4446-ad0b-ffaf2a35ab5d None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:05.105690 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8454c78-9b0f-4446-ad0b-ffaf2a35ab5d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:05.111871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76feafee-54e4-4892-b9b8-94b34649e65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:05.113007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76feafee-54e4-4892-b9b8-94b34649e65f None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:05.113543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76feafee-54e4-4892-b9b8-94b34649e65f None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:05.114082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76feafee-54e4-4892-b9b8-94b34649e65f None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:05.115477 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76feafee-54e4-4892-b9b8-94b34649e65f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:05.116149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76feafee-54e4-4892-b9b8-94b34649e65f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:05.117655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76feafee-54e4-4892-b9b8-94b34649e65f None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:05.121382 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8454c78-9b0f-4446-ad0b-ffaf2a35ab5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:05.126415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-e8454c78-9b0f-4446-ad0b-ffaf2a35ab5d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:59:05.126415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:05.126415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:05.126415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:05.126415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:05.126415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:05.126415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:05.126415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:05.126415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:05.126415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:05.126415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:05.126415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:05.126415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:05.126415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:05.126415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:05.126415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:05.127400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:05.127400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:05.127400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:05.127400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:05.127400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:05.127400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:05.127400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:05.127400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:05.127400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:05.127400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:05.127400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:05.127400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:05.127400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:05.127400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:05.127400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:05.127400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:05.128268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:05.128268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:05.128268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:05.128268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:05.128268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:05.128268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:05.128268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:05.128268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:05.128268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:05.128268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:05.128268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:59:05.128268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:05.128268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 456/905] 10.222.0.93 () {56 vars in 1120 bytes} [Mon Aug 1 14:59:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:05.130036 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 449/905] 10.222.0.93 () {56 vars in 1126 bytes} [Mon Aug 1 14:59:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:05.133692 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8454c78-9b0f-4446-ad0b-ffaf2a35ab5d None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:05.134967 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76feafee-54e4-4892-b9b8-94b34649e65f None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:05.215608 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:05.255081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76feafee-54e4-4892-b9b8-94b34649e65f None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:05.289922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8454c78-9b0f-4446-ad0b-ffaf2a35ab5d None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:05.293179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a3131-6fb9-4bac-9d91-963354d95b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:05.293801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a3131-6fb9-4bac-9d91-963354d95b80 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:05.294075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a3131-6fb9-4bac-9d91-963354d95b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:05.294345 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a3131-6fb9-4bac-9d91-963354d95b80 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:05.295096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a3131-6fb9-4bac-9d91-963354d95b80 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:05.295547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a3131-6fb9-4bac-9d91-963354d95b80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:05.297996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a3131-6fb9-4bac-9d91-963354d95b80 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:05.318774 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 450/906] 10.222.0.93 () {56 vars in 1115 bytes} [Mon Aug 1 14:59:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:05.322458 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a3131-6fb9-4bac-9d91-963354d95b80 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:05.337261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01b727-143d-4caa-a7cb-c36036c6d31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:05.338309 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01b727-143d-4caa-a7cb-c36036c6d31a None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:05.338882 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01b727-143d-4caa-a7cb-c36036c6d31a None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:05.339241 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01b727-143d-4caa-a7cb-c36036c6d31a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:05.340771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01b727-143d-4caa-a7cb-c36036c6d31a None admin] RBAC: Adding query filter params (interface=public, service_id=59c6d6ad190145128d77069cbbcc371c, region_id=RegionOne) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:05.341083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01b727-143d-4caa-a7cb-c36036c6d31a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:05.342489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01b727-143d-4caa-a7cb-c36036c6d31a None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:05.350225 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 457/907] 10.222.0.93 () {56 vars in 1251 bytes} [Mon Aug 1 14:59:05 2022] GET /identity/v3/endpoints?service_id=59c6d6ad190145128d77069cbbcc371c&interface=public®ion_id=RegionOne => generated 194 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:05.358767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01b727-143d-4caa-a7cb-c36036c6d31a None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:05.388400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:05.467535 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a3131-6fb9-4bac-9d91-963354d95b80 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:05.484880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01b727-143d-4caa-a7cb-c36036c6d31a None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:05.497485 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:05.517492 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30a14e-8a10-4b49-8d28-40c91cd46a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:05.518575 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30a14e-8a10-4b49-8d28-40c91cd46a77 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:05.518963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30a14e-8a10-4b49-8d28-40c91cd46a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:05.519352 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30a14e-8a10-4b49-8d28-40c91cd46a77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:05.531462 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fc886-ebaa-49e1-a4eb-19748d961667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:05.532978 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fc886-ebaa-49e1-a4eb-19748d961667 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:05.533425 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fc886-ebaa-49e1-a4eb-19748d961667 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:05.533587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30a14e-8a10-4b49-8d28-40c91cd46a77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:05.533986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fc886-ebaa-49e1-a4eb-19748d961667 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:05.534779 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13fc886-ebaa-49e1-a4eb-19748d961667 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:05.535350 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30a14e-8a10-4b49-8d28-40c91cd46a77 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:05.536650 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13fc886-ebaa-49e1-a4eb-19748d961667 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:05.544856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 458/908] 10.222.0.93 () {56 vars in 1087 bytes} [Mon Aug 1 14:59:05 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:59:05.558318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-db30a14e-8a10-4b49-8d28-40c91cd46a77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:05.558318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:05.558318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:05.558318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:05.558318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:05.558318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:05.558318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:05.558318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:05.558318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:05.558318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:05.558318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:05.558318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:05.558318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:59:05.558318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:59:05.558318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:59:05.559509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:59:05.559509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:05.559509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:05.559509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:05.559509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:05.559509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:05.559509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:05.559509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:05.559509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:05.559509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:05.559509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:05.559509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:05.559509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:05.559509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:05.559509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:05.559509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:05.559509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:05.560572 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:59:05.560572 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:59:05.560572 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:05.560572 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:05.562589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 451/909] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:05.569145 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fc886-ebaa-49e1-a4eb-19748d961667 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:05.679883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:05.700799 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fc886-ebaa-49e1-a4eb-19748d961667 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:05.749210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0bd84-9041-4d06-a602-f09b7e1c57d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:05.749785 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0bd84-9041-4d06-a602-f09b7e1c57d9 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:05.750068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0bd84-9041-4d06-a602-f09b7e1c57d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:05.750346 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0bd84-9041-4d06-a602-f09b7e1c57d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:05.751944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b0bd84-9041-4d06-a602-f09b7e1c57d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:05.751944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b0bd84-9041-4d06-a602-f09b7e1c57d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:05.752928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b0bd84-9041-4d06-a602-f09b7e1c57d9 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:05.767607 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 459/910] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:59:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:05.774158 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30a14e-8a10-4b49-8d28-40c91cd46a77 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:05.914301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30a14e-8a10-4b49-8d28-40c91cd46a77 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:05.957675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f097ae-8b57-4309-b411-460ea5d2e7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:05.958635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f097ae-8b57-4309-b411-460ea5d2e7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:05.959120 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f097ae-8b57-4309-b411-460ea5d2e7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:05.959574 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f097ae-8b57-4309-b411-460ea5d2e7b9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:05.969119 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f097ae-8b57-4309-b411-460ea5d2e7b9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:05.970943 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f097ae-8b57-4309-b411-460ea5d2e7b9 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:05.986874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-a1f097ae-8b57-4309-b411-460ea5d2e7b9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:59:05.986874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:05.986874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:05.986874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:05.986874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:05.986874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:05.986874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:05.986874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:05.986874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:05.986874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:05.986874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:05.986874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:05.986874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:59:05.986874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:59:05.986874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:59:05.986874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:59:05.987916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:05.987916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:05.987916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:59:05.987916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:59:05.987916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:59:05.987916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:59:05.987916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:05.987916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:05.987916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:05.987916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:05.987916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:05.987916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:05.987916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:05.987916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:05.987916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:05.987916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:05.987916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:05.987916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:05.989557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:05.989557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:05.989557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:59:05.989557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:59:05.989557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:59:05.989557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:59:05.989557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:59:05.989557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:59:05.989557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:59:05.989557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:05.992174 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 452/911] 10.222.0.93 () {56 vars in 1117 bytes} [Mon Aug 1 14:59:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:05.998348 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0bd84-9041-4d06-a602-f09b7e1c57d9 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:06.122818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0bd84-9041-4d06-a602-f09b7e1c57d9 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:06.167156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4bdb75-d867-481d-a6ea-88980f6ce4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:06.168229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bdb75-d867-481d-a6ea-88980f6ce4fe None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:06.168807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bdb75-d867-481d-a6ea-88980f6ce4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:06.169256 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bdb75-d867-481d-a6ea-88980f6ce4fe None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:06.170146 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4bdb75-d867-481d-a6ea-88980f6ce4fe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:06.170459 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4bdb75-d867-481d-a6ea-88980f6ce4fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:06.172623 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4bdb75-d867-481d-a6ea-88980f6ce4fe None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:06.201990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 460/912] 10.222.0.93 () {56 vars in 1149 bytes} [Mon Aug 1 14:59:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:06.208255 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f097ae-8b57-4309-b411-460ea5d2e7b9 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:06.260628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:06.346895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f097ae-8b57-4309-b411-460ea5d2e7b9 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:06.387647 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32c806-3990-4bed-9d6d-9e9613a4dc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:06.388406 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32c806-3990-4bed-9d6d-9e9613a4dc6a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:06.388746 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32c806-3990-4bed-9d6d-9e9613a4dc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:06.389055 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32c806-3990-4bed-9d6d-9e9613a4dc6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:06.466140 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32c806-3990-4bed-9d6d-9e9613a4dc6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:06.467543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32c806-3990-4bed-9d6d-9e9613a4dc6a None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:06.520879 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:06.625838 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 453/913] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:59:06 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:59:06.632065 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4bdb75-d867-481d-a6ea-88980f6ce4fe None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:06.637242 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:06.769923 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4bdb75-d867-481d-a6ea-88980f6ce4fe None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:06.827266 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d25d97-281b-42f1-b28d-f72a62e952de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:06.828347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d25d97-281b-42f1-b28d-f72a62e952de None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:06.828787 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d25d97-281b-42f1-b28d-f72a62e952de None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:06.829115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d25d97-281b-42f1-b28d-f72a62e952de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:06.842536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d25d97-281b-42f1-b28d-f72a62e952de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:06.844936 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d25d97-281b-42f1-b28d-f72a62e952de None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:06.867908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-b3d25d97-281b-42f1-b28d-f72a62e952de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:06.867908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:06.867908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:06.867908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:06.867908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:06.867908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:06.867908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:06.867908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:06.867908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:06.867908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:06.867908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:06.867908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:06.867908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:59:06.867908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:59:06.867908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:59:06.870297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:59:06.870297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:06.870297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:06.870297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:06.870297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:06.870297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:06.870297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:06.870297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:06.870297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:06.870297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:06.870297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:06.870297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:06.870297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:06.870297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:06.870297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:06.870297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:06.870297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:06.872641 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:59:06.872641 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:59:06.872641 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:06.872641 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:06.872641 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 461/914] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:06.877720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32c806-3990-4bed-9d6d-9e9613a4dc6a None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:06.927024 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abd74b-634a-44ba-8bda-161bb17a1767 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:06.927507 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abd74b-634a-44ba-8bda-161bb17a1767 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:06.927932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abd74b-634a-44ba-8bda-161bb17a1767 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:06.931621 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 462/915] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:06.937719 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60173734-9da6-46c4-a212-a8c0847e106e None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:06.938033 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60173734-9da6-46c4-a212-a8c0847e106e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:06.938312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60173734-9da6-46c4-a212-a8c0847e106e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:06.960582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-60173734-9da6-46c4-a212-a8c0847e106e None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:06.962067 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-60173734-9da6-46c4-a212-a8c0847e106e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:07.044423 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32c806-3990-4bed-9d6d-9e9613a4dc6a None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:07.061929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:07.106500 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af1be8-ea6b-4d9a-90cd-baa55b6ac68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:07.107943 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af1be8-ea6b-4d9a-90cd-baa55b6ac68e None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:07.108656 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 463/916] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:59:07.108972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af1be8-ea6b-4d9a-90cd-baa55b6ac68e None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:07.109806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af1be8-ea6b-4d9a-90cd-baa55b6ac68e None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:07.111368 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af1be8-ea6b-4d9a-90cd-baa55b6ac68e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:07.112181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af1be8-ea6b-4d9a-90cd-baa55b6ac68e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:07.115187 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af1be8-ea6b-4d9a-90cd-baa55b6ac68e None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:07.124729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:07.134792 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60173734-9da6-46c4-a212-a8c0847e106e None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:07.139887 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 454/917] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:07.146302 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af1be8-ea6b-4d9a-90cd-baa55b6ac68e None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:07.264122 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60173734-9da6-46c4-a212-a8c0847e106e None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:07.264379 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af1be8-ea6b-4d9a-90cd-baa55b6ac68e None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:07.308895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49de15-22ef-4d27-b993-68450f91fe23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:07.309468 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49de15-22ef-4d27-b993-68450f91fe23 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:07.309762 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49de15-22ef-4d27-b993-68450f91fe23 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:07.310206 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49de15-22ef-4d27-b993-68450f91fe23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:07.313100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0b5cd-cc57-471d-9a61-8392aed2af3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:07.314038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0b5cd-cc57-471d-9a61-8392aed2af3f None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:07.314038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0b5cd-cc57-471d-9a61-8392aed2af3f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:07.314297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0b5cd-cc57-471d-9a61-8392aed2af3f None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:07.315082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c0b5cd-cc57-471d-9a61-8392aed2af3f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:07.316648 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c0b5cd-cc57-471d-9a61-8392aed2af3f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:07.319316 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: WARNING keystone.server.flask.common [None req-03c0b5cd-cc57-471d-9a61-8392aed2af3f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 14:59:07.331097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49de15-22ef-4d27-b993-68450f91fe23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:07.334217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49de15-22ef-4d27-b993-68450f91fe23 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:07.342619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.sql.core [None req-03c0b5cd-cc57-471d-9a61-8392aed2af3f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 14:59:07.342619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 14:59:07.342619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [parameters: {'id': '94417e642ffb4410a0efb4e28eb38a62', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 14:59:07.342619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80412) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 14:59:07.348351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-ca49de15-22ef-4d27-b993-68450f91fe23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:59:07.348351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:07.348351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:07.348351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:07.348351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:07.348351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:07.348351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:07.348351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:07.348351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:07.348351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:07.348351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:07.348351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:07.348351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:59:07.348351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:59:07.348351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:59:07.349455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:59:07.349455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:07.349455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:07.349455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:07.349455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:07.349455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:07.349455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:07.349455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:07.349455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:07.349455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:07.349455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:07.349455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:07.349455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:07.349455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:07.349455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:07.349455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:07.349455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:07.350424 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:59:07.350424 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:59:07.350424 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:59:07.350424 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:59:07.350424 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:59:07.350424 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:59:07.350424 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:59:07.350424 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:07.354595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 455/918] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:59:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:07.355725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc8123-284e-4a06-967d-e1571297c932 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:07.356006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc8123-284e-4a06-967d-e1571297c932 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:07.356277 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc8123-284e-4a06-967d-e1571297c932 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:07.357847 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 456/919] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:07.358338 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc8123-284e-4a06-967d-e1571297c932 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:07.367082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-03c0b5cd-cc57-471d-9a61-8392aed2af3f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 94417e642ffb4410a0efb4e28eb38a62.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 94417e642ffb4410a0efb4e28eb38a62. Aug 01 14:59:07.367082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:07.367082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 14:59:07.367082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 14:59:07.367082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 14:59:07.367082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 14:59:07.367082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 14:59:07.367082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 14:59:07.367082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 14:59:07.367082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application conn.query(q) Aug 01 14:59:07.367082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 14:59:07.367082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 14:59:07.367082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 14:59:07.370726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application result.read() Aug 01 14:59:07.370726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 14:59:07.370726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 14:59:07.370726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 14:59:07.370726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 14:59:07.370726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 14:59:07.370726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 14:59:07.370726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 14:59:07.370726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 14:59:07.370726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 14:59:07.370726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:07.370726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 14:59:07.370726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:07.370726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:07.370726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 14:59:07.370726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 14:59:07.370726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 01 14:59:07.370726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 01 14:59:07.375962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 14:59:07.375962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application next(self.gen) Aug 01 14:59:07.375962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 14:59:07.375962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application yield resource Aug 01 14:59:07.375962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 14:59:07.375962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application next(self.gen) Aug 01 14:59:07.375962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 14:59:07.375962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 14:59:07.375962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 14:59:07.375962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 14:59:07.375962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 14:59:07.375962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise self.value Aug 01 14:59:07.375962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 14:59:07.375962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 14:59:07.375962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 14:59:07.375962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application session.commit() Aug 01 14:59:07.375962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 14:59:07.381075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 14:59:07.381075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 14:59:07.381075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 14:59:07.381075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 14:59:07.381075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self.session.flush() Aug 01 14:59:07.381075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 14:59:07.381075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 14:59:07.381075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 14:59:07.381075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 14:59:07.381075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 14:59:07.381075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application compat.raise_( Aug 01 14:59:07.381075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 14:59:07.381075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception Aug 01 14:59:07.381075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 14:59:07.381075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 14:59:07.381075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 14:59:07.381075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 14:59:07.386240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 14:59:07.386240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 14:59:07.386240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 14:59:07.386240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 14:59:07.386240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 14:59:07.386240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 14:59:07.386240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 14:59:07.386240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 14:59:07.386240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 14:59:07.386240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 14:59:07.386240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 14:59:07.386240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 14:59:07.386240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 14:59:07.386240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 14:59:07.386240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 14:59:07.389928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 14:59:07.389928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 14:59:07.389928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception Aug 01 14:59:07.389928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 14:59:07.389928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 14:59:07.389928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 14:59:07.389928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 14:59:07.389928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 14:59:07.389928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 14:59:07.389928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 14:59:07.389928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application conn.query(q) Aug 01 14:59:07.389928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 14:59:07.389928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 14:59:07.389928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 14:59:07.389928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application result.read() Aug 01 14:59:07.389928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 14:59:07.389928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 14:59:07.393220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 14:59:07.393220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 14:59:07.393220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 14:59:07.393220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 14:59:07.393220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 14:59:07.393220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 14:59:07.393220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 14:59:07.393220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 14:59:07.393220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [parameters: {'id': '94417e642ffb4410a0efb4e28eb38a62', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 14:59:07.393220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 14:59:07.393220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:07.393220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:59:07.393220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:07.393220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:07.393220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 01 14:59:07.399252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 01 14:59:07.399252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 14:59:07.399252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 14:59:07.399252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 01 14:59:07.399252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:07.399252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:59:07.399252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:07.399252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:07.399252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:07.399252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:07.399252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:07.399252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:07.399252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:07.399252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:07.399252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:07.399252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:07.399252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:07.402962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:07.402962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 01 14:59:07.402962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 01 14:59:07.402962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:07.402962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:07.402962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 01 14:59:07.402962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 01 14:59:07.402962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 94417e642ffb4410a0efb4e28eb38a62. Aug 01 14:59:07.402962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:07.402962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 464/920] 10.222.0.93 () {60 vars in 1140 bytes} [Mon Aug 1 14:59:07 2022] POST /identity/v3/projects => generated 262 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:07.402962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc111188-5e6e-47ad-8ae0-2acb05f2fd18 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:07.402962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc111188-5e6e-47ad-8ae0-2acb05f2fd18 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:07.406290 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc111188-5e6e-47ad-8ae0-2acb05f2fd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:07.406290 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-dc111188-5e6e-47ad-8ae0-2acb05f2fd18 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:07.406290 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-dc111188-5e6e-47ad-8ae0-2acb05f2fd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:07.440251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:07.448616 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:07.488519 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 465/921] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:07.488519 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc111188-5e6e-47ad-8ae0-2acb05f2fd18 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:07.544126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc8123-284e-4a06-967d-e1571297c932 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:07.604812 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34189a-cb86-42cc-acc4-e9cfb5f3beef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:07.605490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34189a-cb86-42cc-acc4-e9cfb5f3beef None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:07.605766 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34189a-cb86-42cc-acc4-e9cfb5f3beef None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:07.606042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34189a-cb86-42cc-acc4-e9cfb5f3beef None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:07.606793 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34189a-cb86-42cc-acc4-e9cfb5f3beef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:07.607107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34189a-cb86-42cc-acc4-e9cfb5f3beef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:07.608611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34189a-cb86-42cc-acc4-e9cfb5f3beef None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:07.630658 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 457/922] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:59:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:07.632530 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34189a-cb86-42cc-acc4-e9cfb5f3beef None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:07.694681 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc111188-5e6e-47ad-8ae0-2acb05f2fd18 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:07.759257 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bed6b-c163-46e4-99d4-d162469ca4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:07.760154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bed6b-c163-46e4-99d4-d162469ca4bf None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:07.760633 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bed6b-c163-46e4-99d4-d162469ca4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:07.760968 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bed6b-c163-46e4-99d4-d162469ca4bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:07.775457 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022bed6b-c163-46e4-99d4-d162469ca4bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:07.778395 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022bed6b-c163-46e4-99d4-d162469ca4bf None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:07.796445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-022bed6b-c163-46e4-99d4-d162469ca4bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:59:07.796445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:07.796445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:07.796445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:07.796445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:07.796445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:07.796445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:07.796445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:07.796445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:07.796445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:07.796445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:07.796445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:07.796445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:59:07.796445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:59:07.796445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:59:07.797777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:59:07.797777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:07.797777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:07.797777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:07.797777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:07.797777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:07.797777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:07.797777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:07.797777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:07.797777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:07.797777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:07.797777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:07.797777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:07.797777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:07.797777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:07.797777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:07.797777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:07.799482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:59:07.799482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:59:07.799482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:59:07.799482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:59:07.799482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:59:07.799482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:59:07.799482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:59:07.799482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:07.799482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 466/923] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:59:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:07.799482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bed6b-c163-46e4-99d4-d162469ca4bf None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:07.807539 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34189a-cb86-42cc-acc4-e9cfb5f3beef None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:07.830290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:07.841358 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:07.851259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dba09f-fb66-44bc-838b-3dd220b09594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:07.851900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dba09f-fb66-44bc-838b-3dd220b09594 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:07.852180 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dba09f-fb66-44bc-838b-3dd220b09594 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:07.852448 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dba09f-fb66-44bc-838b-3dd220b09594 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:07.853095 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dba09f-fb66-44bc-838b-3dd220b09594 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:07.854542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dba09f-fb66-44bc-838b-3dd220b09594 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:07.864404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-50dba09f-fb66-44bc-838b-3dd220b09594 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:59:07.864404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:07.864404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:59:07.864404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:59:07.864404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:59:07.864404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:59:07.864404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:59:07.864404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:07.864404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:59:07.864404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:07.864404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:59:07.864404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:07.864404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:07.864404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:07.864404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:07.864404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:07.864404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:07.866414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:07.866414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:07.866414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:07.866414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:07.866414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:07.866414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:07.866414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:59:07.866414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:59:07.866414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:59:07.866414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:59:07.866414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:07.866414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:07.866414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:07.866414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:07.866414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:07.866414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:07.868089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:07.868089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:07.868089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:07.868089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:07.868089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:07.868089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:07.868089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:07.868089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:07.868089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:07.868089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:07.868089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:59:07.868089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:07.868089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:59:07.868089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:07.868089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 458/924] 10.222.0.93 () {56 vars in 1126 bytes} [Mon Aug 1 14:59:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:07.870003 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dba09f-fb66-44bc-838b-3dd220b09594 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:07.931019 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bed6b-c163-46e4-99d4-d162469ca4bf None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:07.982861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02e934-1d8e-4e93-9472-7664f99535c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:07.983608 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02e934-1d8e-4e93-9472-7664f99535c9 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:07.984019 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02e934-1d8e-4e93-9472-7664f99535c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:07.984416 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02e934-1d8e-4e93-9472-7664f99535c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:07.985490 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02e934-1d8e-4e93-9472-7664f99535c9 None admin] RBAC: Adding query filter params (role.id=66135f6413e94a77b7c0762b52a4d38e, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=4370f8f474c2446e989d0d8c5b75e87a) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:07.985912 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02e934-1d8e-4e93-9472-7664f99535c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:07.987559 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02e934-1d8e-4e93-9472-7664f99535c9 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:08.000985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 467/925] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:59:07 2022] GET /identity/v3/role_assignments?user.id=4370f8f474c2446e989d0d8c5b75e87a&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=66135f6413e94a77b7c0762b52a4d38e => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:08.002706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73852c8-8807-4a28-a76b-587bf7368172 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:08.003082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73852c8-8807-4a28-a76b-587bf7368172 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:08.003437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73852c8-8807-4a28-a76b-587bf7368172 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:08.006231 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 468/926] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:08.007479 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73852c8-8807-4a28-a76b-587bf7368172 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:08.035878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dba09f-fb66-44bc-838b-3dd220b09594 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:08.100791 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5963a09-9db3-4012-b6f3-e7ce1cf2177a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:08.101610 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5963a09-9db3-4012-b6f3-e7ce1cf2177a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:08.102022 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5963a09-9db3-4012-b6f3-e7ce1cf2177a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:08.102483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5963a09-9db3-4012-b6f3-e7ce1cf2177a None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:08.103706 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5963a09-9db3-4012-b6f3-e7ce1cf2177a None admin] RBAC: Adding query filter params (name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:08.104237 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5963a09-9db3-4012-b6f3-e7ce1cf2177a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:08.106804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5963a09-9db3-4012-b6f3-e7ce1cf2177a None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:08.137208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 459/927] 10.222.0.93 () {56 vars in 1112 bytes} [Mon Aug 1 14:59:07 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:08.138577 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93906d-18a9-4f5d-aa7d-3da73cb68401 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:08.138946 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93906d-18a9-4f5d-aa7d-3da73cb68401 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:08.139315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93906d-18a9-4f5d-aa7d-3da73cb68401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:08.174150 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-1c93906d-18a9-4f5d-aa7d-3da73cb68401 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:08.176911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-1c93906d-18a9-4f5d-aa7d-3da73cb68401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:08.225523 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:08.242213 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73852c8-8807-4a28-a76b-587bf7368172 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:08.255448 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:08.314353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a631c-ad10-4bbb-8e11-dedcc894e6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:08.315454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a631c-ad10-4bbb-8e11-dedcc894e6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:08.317003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a631c-ad10-4bbb-8e11-dedcc894e6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:08.317206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a631c-ad10-4bbb-8e11-dedcc894e6c1 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:08.318425 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a631c-ad10-4bbb-8e11-dedcc894e6c1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:08.318941 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a631c-ad10-4bbb-8e11-dedcc894e6c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:08.321540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a631c-ad10-4bbb-8e11-dedcc894e6c1 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:08.331594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 469/928] 10.222.0.93 () {56 vars in 1122 bytes} [Mon Aug 1 14:59:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:08.337285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 460/929] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:08.339415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a631c-ad10-4bbb-8e11-dedcc894e6c1 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:08.368814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93906d-18a9-4f5d-aa7d-3da73cb68401 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:08.492101 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a631c-ad10-4bbb-8e11-dedcc894e6c1 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:08.552907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33699ed6-2dfc-4860-9787-21277fea653f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:08.553571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33699ed6-2dfc-4860-9787-21277fea653f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:08.553850 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33699ed6-2dfc-4860-9787-21277fea653f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:08.554166 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33699ed6-2dfc-4860-9787-21277fea653f None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:08.554887 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33699ed6-2dfc-4860-9787-21277fea653f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:08.555196 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33699ed6-2dfc-4860-9787-21277fea653f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:08.556706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33699ed6-2dfc-4860-9787-21277fea653f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:08.563992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 470/930] 10.222.0.93 () {56 vars in 1122 bytes} [Mon Aug 1 14:59:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:08.573868 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33699ed6-2dfc-4860-9787-21277fea653f None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:08.581138 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93906d-18a9-4f5d-aa7d-3da73cb68401 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:08.625153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb9fa1-d871-4fb0-a8be-5f0dffbd2d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:08.625813 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb9fa1-d871-4fb0-a8be-5f0dffbd2d27 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:08.626102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb9fa1-d871-4fb0-a8be-5f0dffbd2d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:08.626351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb9fa1-d871-4fb0-a8be-5f0dffbd2d27 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:08.626977 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb9fa1-d871-4fb0-a8be-5f0dffbd2d27 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:08.627785 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:08.629055 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb9fa1-d871-4fb0-a8be-5f0dffbd2d27 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:08.646823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-81bb9fa1-d871-4fb0-a8be-5f0dffbd2d27 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 14:59:08.646823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:08.646823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:59:08.646823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:59:08.646823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:59:08.646823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:59:08.646823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:59:08.646823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:08.646823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 14:59:08.646823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:08.646823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:59:08.646823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:08.646823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:08.646823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:08.646823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:08.646823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:08.649007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:08.649007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:08.649007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:08.649007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:08.649007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:08.649007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:08.649007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:08.649007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:59:08.649007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:59:08.649007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:59:08.649007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:59:08.649007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:08.649007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:08.649007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:08.649007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:08.649007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:08.650741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:08.650741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:08.650741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:08.650741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:08.650741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:08.650741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:08.650741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:08.650741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:08.650741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:08.650741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:08.650741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:08.650741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:59:08.650741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:08.650741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 14:59:08.650741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:08.656117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 461/931] 10.222.0.93 () {56 vars in 1132 bytes} [Mon Aug 1 14:59:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:08.666521 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb9fa1-d871-4fb0-a8be-5f0dffbd2d27 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:08.719001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:08.786280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33699ed6-2dfc-4860-9787-21277fea653f None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:08.830927 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7f598-cac5-4f01-ab83-47c586c09b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:08.832048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7f598-cac5-4f01-ab83-47c586c09b6c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:08.832594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7f598-cac5-4f01-ab83-47c586c09b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:08.833079 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7f598-cac5-4f01-ab83-47c586c09b6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:08.834068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7f598-cac5-4f01-ab83-47c586c09b6c None admin] RBAC: Adding query filter params (interface=public, service_id=0a1e4f60991547339a2b35c684dfb256, region_id=RegionOne) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:08.834642 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7f598-cac5-4f01-ab83-47c586c09b6c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:08.836409 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7f598-cac5-4f01-ab83-47c586c09b6c None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:08.843012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 471/932] 10.222.0.93 () {56 vars in 1251 bytes} [Mon Aug 1 14:59:08 2022] GET /identity/v3/endpoints?service_id=0a1e4f60991547339a2b35c684dfb256&interface=public®ion_id=RegionOne => generated 194 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:08.851894 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7f598-cac5-4f01-ab83-47c586c09b6c None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:08.852108 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb9fa1-d871-4fb0-a8be-5f0dffbd2d27 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:08.917977 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440abe5-12c4-4151-933c-e225dd703b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:08.919053 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440abe5-12c4-4151-933c-e225dd703b48 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:08.919560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440abe5-12c4-4151-933c-e225dd703b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:08.920038 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440abe5-12c4-4151-933c-e225dd703b48 None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:08.921327 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0440abe5-12c4-4151-933c-e225dd703b48 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:08.921847 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0440abe5-12c4-4151-933c-e225dd703b48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:08.924565 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0440abe5-12c4-4151-933c-e225dd703b48 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:08.934192 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 462/933] 10.222.0.93 () {56 vars in 1126 bytes} [Mon Aug 1 14:59:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:08.944503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440abe5-12c4-4151-933c-e225dd703b48 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:08.960358 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:08.979006 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7f598-cac5-4f01-ab83-47c586c09b6c None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:09.018816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedfa367-fffa-4531-baaa-18af9d73cd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:09.019971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedfa367-fffa-4531-baaa-18af9d73cd79 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:09.020252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedfa367-fffa-4531-baaa-18af9d73cd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:09.020553 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedfa367-fffa-4531-baaa-18af9d73cd79 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:09.021200 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedfa367-fffa-4531-baaa-18af9d73cd79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:09.022606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedfa367-fffa-4531-baaa-18af9d73cd79 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:09.029917 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 472/934] 10.222.0.93 () {56 vars in 1087 bytes} [Mon Aug 1 14:59:08 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:59:09.147805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:09.154689 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440abe5-12c4-4151-933c-e225dd703b48 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:09.203970 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c503321-1f2a-48e3-a346-6860b84dd0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:09.205026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c503321-1f2a-48e3-a346-6860b84dd0ba None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:09.205588 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c503321-1f2a-48e3-a346-6860b84dd0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:09.206084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c503321-1f2a-48e3-a346-6860b84dd0ba None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:09.207638 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c503321-1f2a-48e3-a346-6860b84dd0ba None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:09.208279 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c503321-1f2a-48e3-a346-6860b84dd0ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:09.209971 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c503321-1f2a-48e3-a346-6860b84dd0ba None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:09.216285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 463/935] 10.222.0.93 () {56 vars in 1126 bytes} [Mon Aug 1 14:59:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:09.223097 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedfa367-fffa-4531-baaa-18af9d73cd79 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:09.348112 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedfa367-fffa-4531-baaa-18af9d73cd79 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:09.392137 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1767f7-90df-49f0-a7f9-77d9f0d3244f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:09.393471 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1767f7-90df-49f0-a7f9-77d9f0d3244f None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:09.393960 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1767f7-90df-49f0-a7f9-77d9f0d3244f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:09.395137 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1767f7-90df-49f0-a7f9-77d9f0d3244f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:09.396453 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1767f7-90df-49f0-a7f9-77d9f0d3244f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:09.398303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1767f7-90df-49f0-a7f9-77d9f0d3244f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:09.422873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 473/936] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 14:59:09 2022] POST /identity/v3/endpoints => generated 347 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:59:10.718197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03770f12-2745-4e70-a9b9-bf1b77068f3b None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:10.718968 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03770f12-2745-4e70-a9b9-bf1b77068f3b None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:10.718968 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03770f12-2745-4e70-a9b9-bf1b77068f3b None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:10.720549 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 464/937] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:59:10.726508 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9bf3c-da1f-457a-913f-9cd80864603b None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:10.727290 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9bf3c-da1f-457a-913f-9cd80864603b None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:10.727290 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9bf3c-da1f-457a-913f-9cd80864603b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:10.742313 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:10.752320 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-3df9bf3c-da1f-457a-913f-9cd80864603b None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:10.755610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3df9bf3c-da1f-457a-913f-9cd80864603b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:10.898094 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 474/938] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:10.941667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03770f12-2745-4e70-a9b9-bf1b77068f3b None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:11.061626 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03770f12-2745-4e70-a9b9-bf1b77068f3b None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:11.101272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659c952-aca6-446d-8b16-155f48c0c48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:11.102271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659c952-aca6-446d-8b16-155f48c0c48b None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:11.102756 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659c952-aca6-446d-8b16-155f48c0c48b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:11.103252 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659c952-aca6-446d-8b16-155f48c0c48b None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:11.104153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659c952-aca6-446d-8b16-155f48c0c48b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:11.105941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659c952-aca6-446d-8b16-155f48c0c48b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:11.108973 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: WARNING keystone.server.flask.common [None req-c659c952-aca6-446d-8b16-155f48c0c48b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 14:59:11.125231 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.sql.core [None req-c659c952-aca6-446d-8b16-155f48c0c48b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 14:59:11.125231 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 14:59:11.125231 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [parameters: {'user_id': 'ab9e376cd9b84cc8b5d458177c62075c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 14:59:11.125231 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80411) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 14:59:11.133540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-c659c952-aca6-446d-8b16-155f48c0c48b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 14:59:11.133540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:11.133540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 14:59:11.133540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 14:59:11.133540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 14:59:11.133540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 14:59:11.133540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 14:59:11.133540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 14:59:11.133540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 14:59:11.133540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application conn.query(q) Aug 01 14:59:11.133540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 14:59:11.133540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 14:59:11.133540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 14:59:11.133540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application result.read() Aug 01 14:59:11.133540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 14:59:11.134540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 14:59:11.134540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 14:59:11.134540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 14:59:11.134540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 14:59:11.134540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 14:59:11.134540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 14:59:11.134540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 14:59:11.134540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 14:59:11.134540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:11.134540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 14:59:11.134540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:11.134540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:11.134540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 14:59:11.134540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 14:59:11.134540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 01 14:59:11.134540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 01 14:59:11.134540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 14:59:11.134540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application next(self.gen) Aug 01 14:59:11.135722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 14:59:11.135722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application yield resource Aug 01 14:59:11.135722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 14:59:11.135722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application next(self.gen) Aug 01 14:59:11.135722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 14:59:11.135722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 14:59:11.135722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 14:59:11.135722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 14:59:11.135722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 14:59:11.135722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise self.value Aug 01 14:59:11.135722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 14:59:11.135722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 14:59:11.135722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 14:59:11.135722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application session.commit() Aug 01 14:59:11.135722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 14:59:11.135722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 14:59:11.135722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 14:59:11.136677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 14:59:11.136677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 14:59:11.136677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self.session.flush() Aug 01 14:59:11.136677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 14:59:11.136677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 14:59:11.136677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 14:59:11.136677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 14:59:11.136677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 14:59:11.136677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application compat.raise_( Aug 01 14:59:11.136677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 14:59:11.136677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception Aug 01 14:59:11.136677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 14:59:11.136677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 14:59:11.136677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 14:59:11.136677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 14:59:11.136677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 14:59:11.136677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 14:59:11.137620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 14:59:11.137620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 14:59:11.137620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 01 14:59:11.137620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 01 14:59:11.137620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 14:59:11.137620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 14:59:11.137620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 14:59:11.137620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 14:59:11.137620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 14:59:11.137620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 14:59:11.137620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 14:59:11.137620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 14:59:11.137620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 14:59:11.137620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 14:59:11.137620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 14:59:11.137620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception Aug 01 14:59:11.138483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 14:59:11.138483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 14:59:11.138483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 14:59:11.138483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 14:59:11.138483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 14:59:11.138483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 14:59:11.138483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 14:59:11.138483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application conn.query(q) Aug 01 14:59:11.138483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 14:59:11.138483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 14:59:11.138483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 14:59:11.138483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application result.read() Aug 01 14:59:11.138483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 14:59:11.138483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 14:59:11.138483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 14:59:11.138483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 14:59:11.139460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 14:59:11.139460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 14:59:11.139460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 14:59:11.139460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 14:59:11.139460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 14:59:11.139460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 14:59:11.139460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ab9e376cd9b84cc8b5d458177c62075c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 14:59:11.139460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 14:59:11.139460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:11.139460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:59:11.139460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:11.139460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:11.139460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:11.139460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:11.139460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:11.140274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:11.140274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:11.140274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:11.140274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:11.140274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:11.140274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:11.140274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:11.140274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 01 14:59:11.140274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 01 14:59:11.140274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:11.140274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:11.140274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:59:11.140274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:59:11.140274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:59:11.140274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:59:11.140274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 01 14:59:11.140274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 01 14:59:11.143552 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 01 14:59:11.143552 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 01 14:59:11.143552 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 14:59:11.143552 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 14:59:11.143552 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 14:59:11.143552 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:11.147017 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 465/939] 10.222.0.93 () {60 vars in 1131 bytes} [Mon Aug 1 14:59:10 2022] POST /identity/v3/users => generated 160 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:11.156567 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9bf3c-da1f-457a-913f-9cd80864603b None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:11.352342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9bf3c-da1f-457a-913f-9cd80864603b None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:11.384613 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39dc23-9687-4772-a0cf-0c8686a22ce1 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:11.385293 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39dc23-9687-4772-a0cf-0c8686a22ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:11.385797 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39dc23-9687-4772-a0cf-0c8686a22ce1 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:11.395985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 466/940] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:11.398122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cad63-8e88-4076-88e6-d94481c17455 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:11.398660 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cad63-8e88-4076-88e6-d94481c17455 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:11.399263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cad63-8e88-4076-88e6-d94481c17455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:11.412933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:11.422492 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32017a-19fb-4711-92d3-acab2396bc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:11.423691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32017a-19fb-4711-92d3-acab2396bc25 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:11.423894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-888cad63-8e88-4076-88e6-d94481c17455 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:11.424344 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32017a-19fb-4711-92d3-acab2396bc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:11.424964 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32017a-19fb-4711-92d3-acab2396bc25 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:11.426742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-888cad63-8e88-4076-88e6-d94481c17455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:11.441620 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32017a-19fb-4711-92d3-acab2396bc25 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:11.445376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32017a-19fb-4711-92d3-acab2396bc25 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:11.449996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:11.466078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-2b32017a-19fb-4711-92d3-acab2396bc25 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 14:59:11.466078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:11.466078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:11.466078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:11.466078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:11.466078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:11.466078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:11.466078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:11.466078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:11.466078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:11.466078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:11.466078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:11.466078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:59:11.466078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:59:11.466078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:59:11.466078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:59:11.468012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:11.468012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:11.468012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:59:11.468012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:59:11.468012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:59:11.468012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:59:11.468012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:11.468012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:11.468012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:11.468012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:11.468012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:11.468012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:11.468012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:11.468012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:11.468012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:11.468012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:11.468012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:11.468012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:11.469874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:11.469874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:11.469874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:59:11.469874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:59:11.469874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:59:11.469874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:59:11.469874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:59:11.469874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:59:11.469874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 14:59:11.469874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:11.469874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 475/941] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:59:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:11.469874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5999d17-bc50-4bd9-b2bd-987699770eb9 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:11.469874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5999d17-bc50-4bd9-b2bd-987699770eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:11.471284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5999d17-bc50-4bd9-b2bd-987699770eb9 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:11.471284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 476/942] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:11.478203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12490b3-240d-4943-957b-9a484d5cbf1b None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:11.478597 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12490b3-240d-4943-957b-9a484d5cbf1b None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:11.478861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12490b3-240d-4943-957b-9a484d5cbf1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:11.504550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-a12490b3-240d-4943-957b-9a484d5cbf1b None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:11.505952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a12490b3-240d-4943-957b-9a484d5cbf1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:11.563330 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 467/943] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:11.565180 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888cad63-8e88-4076-88e6-d94481c17455 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:11.604675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 477/944] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:59:11.605959 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12490b3-240d-4943-957b-9a484d5cbf1b None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:11.772574 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888cad63-8e88-4076-88e6-d94481c17455 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:11.778349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12490b3-240d-4943-957b-9a484d5cbf1b None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:11.822768 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:11.840021 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0406fa-2f45-424d-8a68-c35876bc6ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:11.840604 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13515c3c-97f5-4080-ae54-6a20bfc78e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:11.841095 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0406fa-2f45-424d-8a68-c35876bc6ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:11.841255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13515c3c-97f5-4080-ae54-6a20bfc78e66 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:11.841524 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13515c3c-97f5-4080-ae54-6a20bfc78e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:11.841675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0406fa-2f45-424d-8a68-c35876bc6ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:11.841809 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13515c3c-97f5-4080-ae54-6a20bfc78e66 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:11.842117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0406fa-2f45-424d-8a68-c35876bc6ae7 None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:11.842427 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13515c3c-97f5-4080-ae54-6a20bfc78e66 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:11.843385 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0406fa-2f45-424d-8a68-c35876bc6ae7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:11.843903 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0406fa-2f45-424d-8a68-c35876bc6ae7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:11.844056 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13515c3c-97f5-4080-ae54-6a20bfc78e66 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:11.846270 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0406fa-2f45-424d-8a68-c35876bc6ae7 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:11.856855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-13515c3c-97f5-4080-ae54-6a20bfc78e66 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:59:11.856855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:11.856855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:59:11.856855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:59:11.856855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:59:11.856855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:59:11.856855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:59:11.856855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:11.856855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:59:11.856855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:11.856855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:59:11.856855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:11.856855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:11.856855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:11.856855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:11.856855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:11.856855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:11.858837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:11.858837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:11.858837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:11.858837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:11.858837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:11.858837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:11.858837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:59:11.858837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:59:11.858837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:59:11.858837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:59:11.858837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:11.858837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:11.858837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:11.858837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:11.858837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:11.858837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:11.860584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:11.860584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:11.860584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:11.860584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:11.860584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:11.860584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:11.860584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:11.860584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:11.860584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:11.860584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:11.860584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:59:11.860584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:11.860584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:59:11.860584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:11.860584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 478/945] 10.222.0.93 () {56 vars in 1126 bytes} [Mon Aug 1 14:59:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:11.866034 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13515c3c-97f5-4080-ae54-6a20bfc78e66 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:11.922342 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 468/946] 10.222.0.93 () {56 vars in 1101 bytes} [Mon Aug 1 14:59:11 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:11.925061 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0406fa-2f45-424d-8a68-c35876bc6ae7 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:11.961761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:12.062293 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13515c3c-97f5-4080-ae54-6a20bfc78e66 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:12.126269 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff76d4-a19f-45dd-9ef6-64d3e584fb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:12.127147 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff76d4-a19f-45dd-9ef6-64d3e584fb8a None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:12.127432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff76d4-a19f-45dd-9ef6-64d3e584fb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:12.127722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff76d4-a19f-45dd-9ef6-64d3e584fb8a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:12.145812 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0406fa-2f45-424d-8a68-c35876bc6ae7 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:12.147181 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff76d4-a19f-45dd-9ef6-64d3e584fb8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:12.157615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-dcff76d4-a19f-45dd-9ef6-64d3e584fb8a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:59:12.157615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:12.157615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:12.157615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:12.157615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:12.157615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:12.157615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:12.157615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:12.157615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:12.157615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:12.157615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:12.157615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:12.157615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:12.157615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:12.157615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:12.157615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:12.159535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:12.159535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:12.159535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:12.159535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:12.159535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:12.159535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:12.159535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:12.159535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:12.159535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:12.159535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:12.159535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:12.159535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:12.159535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:12.159535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:12.159535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:12.159535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:12.162373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:12.162373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:12.162373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:12.162373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:12.162373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:12.162373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:12.162373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:12.162373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:12.162373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:12.162373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:12.162373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:59:12.162373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:12.162373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 479/947] 10.222.0.93 () {56 vars in 1120 bytes} [Mon Aug 1 14:59:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:12.162373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ddacd-685b-4359-8cd8-0aefde8d2507 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:12.164168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ddacd-685b-4359-8cd8-0aefde8d2507 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:12.164168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ddacd-685b-4359-8cd8-0aefde8d2507 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:12.168553 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 480/948] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:12.168844 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ddacd-685b-4359-8cd8-0aefde8d2507 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:12.203286 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0cc8c-b6ad-4241-944f-1d448a060fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:12.203990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0cc8c-b6ad-4241-944f-1d448a060fcc None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:12.204301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0cc8c-b6ad-4241-944f-1d448a060fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:12.205570 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0cc8c-b6ad-4241-944f-1d448a060fcc None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:12.205570 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0cc8c-b6ad-4241-944f-1d448a060fcc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:12.205570 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0cc8c-b6ad-4241-944f-1d448a060fcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:12.207511 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0cc8c-b6ad-4241-944f-1d448a060fcc None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:12.224310 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 469/949] 10.222.0.93 () {56 vars in 1122 bytes} [Mon Aug 1 14:59:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:12.224310 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26666e60-3fb2-4e6d-a0c5-862360fd5f36 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:12.224310 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26666e60-3fb2-4e6d-a0c5-862360fd5f36 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:12.224310 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26666e60-3fb2-4e6d-a0c5-862360fd5f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:12.241808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-26666e60-3fb2-4e6d-a0c5-862360fd5f36 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:12.243616 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-26666e60-3fb2-4e6d-a0c5-862360fd5f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:12.256541 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:12.330613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ddacd-685b-4359-8cd8-0aefde8d2507 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:12.336281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 470/950] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:12.337183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26666e60-3fb2-4e6d-a0c5-862360fd5f36 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:12.346044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:12.376011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e72a0-26ea-460c-b38f-aa749b9ffe77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:12.376974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e72a0-26ea-460c-b38f-aa749b9ffe77 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:12.377261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e72a0-26ea-460c-b38f-aa749b9ffe77 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:12.377530 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e72a0-26ea-460c-b38f-aa749b9ffe77 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:12.378343 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e72a0-26ea-460c-b38f-aa749b9ffe77 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:12.379194 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e72a0-26ea-460c-b38f-aa749b9ffe77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:12.380691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e72a0-26ea-460c-b38f-aa749b9ffe77 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:12.391431 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 481/951] 10.222.0.93 () {56 vars in 1115 bytes} [Mon Aug 1 14:59:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:12.392077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e72a0-26ea-460c-b38f-aa749b9ffe77 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:12.470968 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26666e60-3fb2-4e6d-a0c5-862360fd5f36 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:12.513838 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab484757-f81c-4eda-8dd5-4599e28bc4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:12.514011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e72a0-26ea-460c-b38f-aa749b9ffe77 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:12.514471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab484757-f81c-4eda-8dd5-4599e28bc4fe None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:12.514754 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab484757-f81c-4eda-8dd5-4599e28bc4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:12.515140 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab484757-f81c-4eda-8dd5-4599e28bc4fe None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:12.516075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab484757-f81c-4eda-8dd5-4599e28bc4fe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:12.516384 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab484757-f81c-4eda-8dd5-4599e28bc4fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:12.517954 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab484757-f81c-4eda-8dd5-4599e28bc4fe None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:12.523385 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 471/952] 10.222.0.93 () {56 vars in 1122 bytes} [Mon Aug 1 14:59:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:12.524216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab484757-f81c-4eda-8dd5-4599e28bc4fe None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:12.526388 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:12.562869 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c1920-8fc7-4908-9189-e29e6c9e5b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:12.564026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c1920-8fc7-4908-9189-e29e6c9e5b64 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:12.564395 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c1920-8fc7-4908-9189-e29e6c9e5b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:12.564882 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c1920-8fc7-4908-9189-e29e6c9e5b64 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:12.565836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c1920-8fc7-4908-9189-e29e6c9e5b64 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:12.568326 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c1920-8fc7-4908-9189-e29e6c9e5b64 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:12.583482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-927c1920-8fc7-4908-9189-e29e6c9e5b64 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 14:59:12.583482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:12.583482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:59:12.583482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:59:12.583482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:59:12.583482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:59:12.583482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:59:12.583482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:12.583482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 14:59:12.583482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:12.583482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:59:12.583482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:12.583482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:12.583482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:12.583482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:12.583482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:12.583482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:12.585282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:12.585282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:12.585282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:12.585282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:12.585282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:12.585282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:12.585282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:59:12.585282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:59:12.585282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:59:12.585282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:59:12.585282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:12.585282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:12.585282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:12.585282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:12.585282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:12.585282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:12.587048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:12.587048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:12.587048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:12.587048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:12.587048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:12.587048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:12.587048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:12.587048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:12.587048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:12.587048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:12.587048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:59:12.587048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:12.587048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 14:59:12.587048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:12.587048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 482/953] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:59:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:12.588725 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c1920-8fc7-4908-9189-e29e6c9e5b64 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:12.628425 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:12.679331 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab484757-f81c-4eda-8dd5-4599e28bc4fe None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:12.722158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c1920-8fc7-4908-9189-e29e6c9e5b64 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:12.743330 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def099d0-f2ff-4f98-86c0-fe3a5e084ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:12.744429 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def099d0-f2ff-4f98-86c0-fe3a5e084ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:12.745216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def099d0-f2ff-4f98-86c0-fe3a5e084ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:12.745717 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def099d0-f2ff-4f98-86c0-fe3a5e084ad2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:12.761609 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a77c7-ab4a-4966-8ea0-a7febf4e0571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:12.762229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a77c7-ab4a-4966-8ea0-a7febf4e0571 None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:12.762507 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a77c7-ab4a-4966-8ea0-a7febf4e0571 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:12.762784 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a77c7-ab4a-4966-8ea0-a7febf4e0571 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:12.763502 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a77c7-ab4a-4966-8ea0-a7febf4e0571 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:12.765064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a77c7-ab4a-4966-8ea0-a7febf4e0571 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:12.765357 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def099d0-f2ff-4f98-86c0-fe3a5e084ad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:12.768168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def099d0-f2ff-4f98-86c0-fe3a5e084ad2 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:12.791141 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 483/954] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 14:59:12 2022] POST /identity/v3/endpoints => generated 349 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:59:12.792050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-def099d0-f2ff-4f98-86c0-fe3a5e084ad2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:12.792050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:12.792050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:12.792050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:12.792050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:12.792050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:12.792050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:12.792050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:12.792050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:12.792050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:12.792050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:12.792050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:12.792050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:59:12.792050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:59:12.792050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:59:12.793484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:59:12.793484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:12.793484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:12.793484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:12.793484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:12.793484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:12.793484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:12.793484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:12.793484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:12.793484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:12.793484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:12.793484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:12.793484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:12.793484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:12.793484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:12.793484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:12.793484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:12.794632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:59:12.794632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:59:12.794632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:12.794632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:12.794882 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a77c7-ab4a-4966-8ea0-a7febf4e0571 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:12.796367 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 472/955] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:12.805126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def099d0-f2ff-4f98-86c0-fe3a5e084ad2 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:12.843275 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:12.953629 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a77c7-ab4a-4966-8ea0-a7febf4e0571 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:12.976380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def099d0-f2ff-4f98-86c0-fe3a5e084ad2 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:13.011927 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3208df-ac11-46b4-a5a9-688df28fbcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:13.013039 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3208df-ac11-46b4-a5a9-688df28fbcfd None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:13.013486 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3208df-ac11-46b4-a5a9-688df28fbcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:13.013932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3208df-ac11-46b4-a5a9-688df28fbcfd None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:13.014964 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3208df-ac11-46b4-a5a9-688df28fbcfd None admin] RBAC: Adding query filter params (name=compute) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:13.015458 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3208df-ac11-46b4-a5a9-688df28fbcfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:13.017909 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3208df-ac11-46b4-a5a9-688df28fbcfd None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:13.020892 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:13.032183 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 484/956] 10.222.0.93 () {56 vars in 1112 bytes} [Mon Aug 1 14:59:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:13.035317 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3208df-ac11-46b4-a5a9-688df28fbcfd None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:13.051041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd1cfe-9385-4688-9498-d787a5288157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:13.052187 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd1cfe-9385-4688-9498-d787a5288157 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:13.052685 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd1cfe-9385-4688-9498-d787a5288157 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:13.053145 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd1cfe-9385-4688-9498-d787a5288157 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:13.054317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd1cfe-9385-4688-9498-d787a5288157 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:13.054787 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd1cfe-9385-4688-9498-d787a5288157 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:13.057100 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd1cfe-9385-4688-9498-d787a5288157 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:13.077444 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 473/957] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:13.089771 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd1cfe-9385-4688-9498-d787a5288157 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:13.162282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:13.179531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3208df-ac11-46b4-a5a9-688df28fbcfd None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:13.227663 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813cad8d-16d8-4c72-97c8-39717c32816b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:13.228551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cad8d-16d8-4c72-97c8-39717c32816b None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:13.229371 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cad8d-16d8-4c72-97c8-39717c32816b None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:13.229752 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cad8d-16d8-4c72-97c8-39717c32816b None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:13.230906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813cad8d-16d8-4c72-97c8-39717c32816b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:13.231437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813cad8d-16d8-4c72-97c8-39717c32816b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:13.233402 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813cad8d-16d8-4c72-97c8-39717c32816b None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:13.249803 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 485/958] 10.222.0.93 () {56 vars in 1112 bytes} [Mon Aug 1 14:59:13 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:13.282159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd1cfe-9385-4688-9498-d787a5288157 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:13.324764 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5dce89-2dba-4449-b3b2-aa1361d35971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:13.325779 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5dce89-2dba-4449-b3b2-aa1361d35971 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:13.326229 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5dce89-2dba-4449-b3b2-aa1361d35971 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:13.326686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5dce89-2dba-4449-b3b2-aa1361d35971 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:13.337348 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5dce89-2dba-4449-b3b2-aa1361d35971 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:13.339337 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5dce89-2dba-4449-b3b2-aa1361d35971 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:13.353547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-2b5dce89-2dba-4449-b3b2-aa1361d35971 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:59:13.353547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:13.353547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:13.353547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:13.353547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:13.353547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:13.353547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:13.353547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:13.353547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:13.353547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:13.353547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:13.353547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:13.353547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:59:13.353547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:59:13.353547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:59:13.353547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:59:13.354823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:13.354823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:13.354823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:59:13.354823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:59:13.354823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:59:13.354823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:59:13.354823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:13.354823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:13.354823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:13.354823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:13.354823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:13.354823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:13.354823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:13.354823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:13.354823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:13.354823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:13.354823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:13.354823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:13.356027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:13.356027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:13.356027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:59:13.356027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:59:13.356027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:59:13.356027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:59:13.356027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:59:13.356027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:59:13.356027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:59:13.356027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:13.358886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 474/959] 10.222.0.93 () {56 vars in 1117 bytes} [Mon Aug 1 14:59:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:13.371574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813cad8d-16d8-4c72-97c8-39717c32816b None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:13.605367 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813cad8d-16d8-4c72-97c8-39717c32816b None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:13.664488 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f1988-195b-470f-90da-d7d83dc0f337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:13.664488 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f1988-195b-470f-90da-d7d83dc0f337 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:13.664488 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f1988-195b-470f-90da-d7d83dc0f337 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:13.664488 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f1988-195b-470f-90da-d7d83dc0f337 None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:13.668482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f1988-195b-470f-90da-d7d83dc0f337 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:13.668482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f1988-195b-470f-90da-d7d83dc0f337 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:13.672488 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f1988-195b-470f-90da-d7d83dc0f337 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:13.703832 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 486/960] 10.222.0.93 () {56 vars in 1149 bytes} [Mon Aug 1 14:59:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:13.710237 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5dce89-2dba-4449-b3b2-aa1361d35971 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:13.801259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:13.917748 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5dce89-2dba-4449-b3b2-aa1361d35971 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:13.986242 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815424f-cce8-4888-8dc3-cf08d17e9925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:13.987290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815424f-cce8-4888-8dc3-cf08d17e9925 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:13.987741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815424f-cce8-4888-8dc3-cf08d17e9925 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:13.988810 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815424f-cce8-4888-8dc3-cf08d17e9925 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:14.008636 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815424f-cce8-4888-8dc3-cf08d17e9925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:14.011338 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815424f-cce8-4888-8dc3-cf08d17e9925 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:14.041032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-6815424f-cce8-4888-8dc3-cf08d17e9925 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:14.041032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:14.041032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:14.041032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:14.041032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:14.041032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:14.041032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:14.041032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:14.041032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:14.041032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:14.041032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:14.041032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:14.041032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:59:14.041032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:59:14.041032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:59:14.042763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:59:14.042763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:14.042763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:14.042763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:14.042763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:14.042763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:14.042763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:14.042763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:14.042763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:14.042763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:14.042763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:14.042763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:14.042763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:14.042763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:14.042763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:14.042763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:14.042763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:14.051029 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:59:14.051029 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:59:14.051029 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:14.051029 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:14.051029 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 475/961] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:14.056618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f1988-195b-470f-90da-d7d83dc0f337 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:14.081919 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:14.217923 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f1988-195b-470f-90da-d7d83dc0f337 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:14.261566 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2ffae-6bad-4036-8446-1a215f3fb0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:14.262194 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2ffae-6bad-4036-8446-1a215f3fb0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:14.262472 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2ffae-6bad-4036-8446-1a215f3fb0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:14.262746 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2ffae-6bad-4036-8446-1a215f3fb0a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:14.263439 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2ffae-6bad-4036-8446-1a215f3fb0a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:14.263740 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2ffae-6bad-4036-8446-1a215f3fb0a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:14.265258 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2ffae-6bad-4036-8446-1a215f3fb0a8 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:14.290057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 487/962] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:14.291622 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815424f-cce8-4888-8dc3-cf08d17e9925 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:14.434645 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:14.445302 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815424f-cce8-4888-8dc3-cf08d17e9925 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:14.509012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba6d5f-bf2c-4353-924a-59d8fd14a1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:14.509708 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba6d5f-bf2c-4353-924a-59d8fd14a1ba None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:14.510129 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba6d5f-bf2c-4353-924a-59d8fd14a1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:14.510318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba6d5f-bf2c-4353-924a-59d8fd14a1ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:14.519041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba6d5f-bf2c-4353-924a-59d8fd14a1ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:14.520814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba6d5f-bf2c-4353-924a-59d8fd14a1ba None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:14.534551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-74ba6d5f-bf2c-4353-924a-59d8fd14a1ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:59:14.534551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:14.534551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:14.534551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:14.534551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:14.534551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:14.534551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:14.534551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:14.534551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:14.534551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:14.534551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:14.534551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:14.534551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:59:14.534551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:59:14.534551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:59:14.535827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:59:14.535827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:14.535827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:14.535827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:14.535827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:14.535827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:14.535827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:14.535827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:14.535827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:14.535827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:14.535827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:14.535827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:14.535827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:14.535827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:14.535827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:14.535827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:14.535827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:14.537054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:59:14.537054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:59:14.537054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:59:14.537054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:59:14.537054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:59:14.537054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:59:14.537054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:59:14.537054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:14.537054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 476/963] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:59:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:14.542147 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2ffae-6bad-4036-8446-1a215f3fb0a8 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:14.649293 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:14.705458 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2ffae-6bad-4036-8446-1a215f3fb0a8 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:14.758050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d1852-4ec5-4f51-9528-b3634ea4b30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:14.759377 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d1852-4ec5-4f51-9528-b3634ea4b30c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:14.759837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d1852-4ec5-4f51-9528-b3634ea4b30c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:14.760307 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d1852-4ec5-4f51-9528-b3634ea4b30c None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:14.761387 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d1852-4ec5-4f51-9528-b3634ea4b30c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:14.761908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d1852-4ec5-4f51-9528-b3634ea4b30c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:14.764033 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d1852-4ec5-4f51-9528-b3634ea4b30c None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:14.791726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 488/964] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:59:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:14.796244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba6d5f-bf2c-4353-924a-59d8fd14a1ba None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:14.944416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba6d5f-bf2c-4353-924a-59d8fd14a1ba None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:14.989908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d341e50-f60f-4596-a312-7d9ce3a18351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:14.990795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d341e50-f60f-4596-a312-7d9ce3a18351 None admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:14.991103 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d341e50-f60f-4596-a312-7d9ce3a18351 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:14.991560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d341e50-f60f-4596-a312-7d9ce3a18351 None admin] PATH_INFO: `/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/4370f8f474c2446e989d0d8c5b75e87a/roles/66135f6413e94a77b7c0762b52a4d38e` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:14.994339 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d341e50-f60f-4596-a312-7d9ce3a18351 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac9f7c74597642b0acecc956429ac4d8, user_id=4370f8f474c2446e989d0d8c5b75e87a, role_id=66135f6413e94a77b7c0762b52a4d38e)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:14.996074 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d341e50-f60f-4596-a312-7d9ce3a18351 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:15.014193 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 477/965] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:59:14 2022] PUT /identity/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/4370f8f474c2446e989d0d8c5b75e87a/roles/66135f6413e94a77b7c0762b52a4d38e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:59:15.255261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa3f7b-b924-43f2-a27f-0a147055a70f None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:15.256028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa3f7b-b924-43f2-a27f-0a147055a70f None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:15.256028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa3f7b-b924-43f2-a27f-0a147055a70f None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:15.257350 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 489/966] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:59:15.263209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7eb2b7-4aa2-47c7-b364-70e21e6167e8 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:15.263522 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7eb2b7-4aa2-47c7-b364-70e21e6167e8 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:15.263774 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7eb2b7-4aa2-47c7-b364-70e21e6167e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:15.283435 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-5e7eb2b7-4aa2-47c7-b364-70e21e6167e8 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:15.285506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-5e7eb2b7-4aa2-47c7-b364-70e21e6167e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:15.297718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:15.377796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 478/967] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:15.418764 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa3f7b-b924-43f2-a27f-0a147055a70f None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:15.610211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543fada-bf57-456a-a920-07921069c804 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:15.610495 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543fada-bf57-456a-a920-07921069c804 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:15.610862 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543fada-bf57-456a-a920-07921069c804 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:15.611519 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa3f7b-b924-43f2-a27f-0a147055a70f None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:15.612604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 479/968] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:15.620120 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb04a4-b783-4c34-80a9-7b56602c12f5 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:15.620638 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb04a4-b783-4c34-80a9-7b56602c12f5 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:15.621067 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb04a4-b783-4c34-80a9-7b56602c12f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:15.647259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-4bdb04a4-b783-4c34-80a9-7b56602c12f5 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:15.648491 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-4bdb04a4-b783-4c34-80a9-7b56602c12f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:15.649906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:15.659071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118a891-b7d3-4d92-beab-135f6c728b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:15.659836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118a891-b7d3-4d92-beab-135f6c728b7c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:15.660115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118a891-b7d3-4d92-beab-135f6c728b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:15.660408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118a891-b7d3-4d92-beab-135f6c728b7c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:15.661058 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118a891-b7d3-4d92-beab-135f6c728b7c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:15.662560 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118a891-b7d3-4d92-beab-135f6c728b7c None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:15.672233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-1118a891-b7d3-4d92-beab-135f6c728b7c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:59:15.672233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:15.672233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:59:15.672233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:59:15.672233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:59:15.672233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:59:15.672233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:59:15.672233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:15.672233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:59:15.672233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:15.672233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:59:15.672233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:15.672233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:15.672233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:15.672233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:15.672233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:15.672233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:15.673841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:15.673841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:15.673841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:15.673841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:15.673841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:15.673841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:15.673841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:59:15.673841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:59:15.673841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:59:15.673841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:59:15.673841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:15.673841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:15.673841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:15.673841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:15.673841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:15.673841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:15.675045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:15.675045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:15.675045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:15.675045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:15.675045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:15.675045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:15.675045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:15.675045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:15.675045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:15.675045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:15.675045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:59:15.675045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:15.675045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:59:15.675045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:15.675045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 490/969] 10.222.0.93 () {56 vars in 1126 bytes} [Mon Aug 1 14:59:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:15.682020 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118a891-b7d3-4d92-beab-135f6c728b7c None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:15.764092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 480/970] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:59:15.799120 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb04a4-b783-4c34-80a9-7b56602c12f5 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:15.848528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118a891-b7d3-4d92-beab-135f6c728b7c None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:15.897600 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed47c60-fa09-4a69-8e71-fba753d3c168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:15.898332 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed47c60-fa09-4a69-8e71-fba753d3c168 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:15.898654 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed47c60-fa09-4a69-8e71-fba753d3c168 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:15.898917 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed47c60-fa09-4a69-8e71-fba753d3c168 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:15.899682 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed47c60-fa09-4a69-8e71-fba753d3c168 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:15.899980 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed47c60-fa09-4a69-8e71-fba753d3c168 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:15.901436 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed47c60-fa09-4a69-8e71-fba753d3c168 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:15.915195 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 491/971] 10.222.0.93 () {56 vars in 1122 bytes} [Mon Aug 1 14:59:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:15.915489 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:15.918498 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed47c60-fa09-4a69-8e71-fba753d3c168 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:15.953570 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb04a4-b783-4c34-80a9-7b56602c12f5 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:16.002253 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64879767-5528-4058-9f05-cfb24f393331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:16.004380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64879767-5528-4058-9f05-cfb24f393331 None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:16.005329 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64879767-5528-4058-9f05-cfb24f393331 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:16.005966 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64879767-5528-4058-9f05-cfb24f393331 None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:16.007655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64879767-5528-4058-9f05-cfb24f393331 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:16.010734 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64879767-5528-4058-9f05-cfb24f393331 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:16.032584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 481/972] 10.222.0.93 () {60 vars in 1141 bytes} [Mon Aug 1 14:59:15 2022] POST /identity/v3/services => generated 242 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:59:16.039768 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:16.105158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed47c60-fa09-4a69-8e71-fba753d3c168 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:16.163407 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee65101-a0cc-4bfa-934d-fc81c24c38e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:16.164310 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee65101-a0cc-4bfa-934d-fc81c24c38e1 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:16.164731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee65101-a0cc-4bfa-934d-fc81c24c38e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:16.165012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee65101-a0cc-4bfa-934d-fc81c24c38e1 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:16.165881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee65101-a0cc-4bfa-934d-fc81c24c38e1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:16.166185 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee65101-a0cc-4bfa-934d-fc81c24c38e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:16.167610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee65101-a0cc-4bfa-934d-fc81c24c38e1 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:16.180513 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 492/973] 10.222.0.93 () {56 vars in 1122 bytes} [Mon Aug 1 14:59:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:16.185969 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64879767-5528-4058-9f05-cfb24f393331 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:16.313380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64879767-5528-4058-9f05-cfb24f393331 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:16.368447 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218db5c1-2368-4d93-b1fe-29c946ffc476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:16.369506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218db5c1-2368-4d93-b1fe-29c946ffc476 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:16.370264 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218db5c1-2368-4d93-b1fe-29c946ffc476 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:16.370762 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218db5c1-2368-4d93-b1fe-29c946ffc476 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:16.372216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218db5c1-2368-4d93-b1fe-29c946ffc476 None admin] RBAC: Adding query filter params (interface=admin, service_id=0a1e4f60991547339a2b35c684dfb256, region_id=RegionOne) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:16.372744 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218db5c1-2368-4d93-b1fe-29c946ffc476 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:16.375383 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218db5c1-2368-4d93-b1fe-29c946ffc476 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:16.379009 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:16.392940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 482/974] 10.222.0.93 () {56 vars in 1249 bytes} [Mon Aug 1 14:59:16 2022] GET /identity/v3/endpoints?service_id=0a1e4f60991547339a2b35c684dfb256&interface=admin®ion_id=RegionOne => generated 193 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:16.399375 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee65101-a0cc-4bfa-934d-fc81c24c38e1 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:16.561498 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee65101-a0cc-4bfa-934d-fc81c24c38e1 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:16.581996 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:16.603809 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc745b8-c24d-469b-aa4d-f76b05ec749d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:16.604504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc745b8-c24d-469b-aa4d-f76b05ec749d None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:16.604794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc745b8-c24d-469b-aa4d-f76b05ec749d None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:16.605068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc745b8-c24d-469b-aa4d-f76b05ec749d None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:16.605729 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc745b8-c24d-469b-aa4d-f76b05ec749d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:16.607170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc745b8-c24d-469b-aa4d-f76b05ec749d None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:16.615847 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 493/975] 10.222.0.93 () {56 vars in 1087 bytes} [Mon Aug 1 14:59:16 2022] GET /identity/v3/services => generated 1727 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:59:17.239084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fc15b-300b-4089-9856-47ba8da4acc9 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:17.239938 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fc15b-300b-4089-9856-47ba8da4acc9 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:17.240078 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fc15b-300b-4089-9856-47ba8da4acc9 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:17.242133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 483/976] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:17.248129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad77d5d-e69a-462e-bf2b-6d4523c4aa11 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:17.248511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad77d5d-e69a-462e-bf2b-6d4523c4aa11 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:17.248714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad77d5d-e69a-462e-bf2b-6d4523c4aa11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:17.270357 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-7ad77d5d-e69a-462e-bf2b-6d4523c4aa11 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:17.272441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-7ad77d5d-e69a-462e-bf2b-6d4523c4aa11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:17.384590 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 494/977] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:59:17.406194 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fc15b-300b-4089-9856-47ba8da4acc9 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:17.538844 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fc15b-300b-4089-9856-47ba8da4acc9 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:17.580675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d66898-0440-49c0-a733-dff2b347c8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:17.581336 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d66898-0440-49c0-a733-dff2b347c8aa None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:17.581659 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d66898-0440-49c0-a733-dff2b347c8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:17.581935 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d66898-0440-49c0-a733-dff2b347c8aa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:17.593168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d66898-0440-49c0-a733-dff2b347c8aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:17.598652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-53d66898-0440-49c0-a733-dff2b347c8aa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:59:17.598652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:17.598652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:17.598652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:17.598652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:17.598652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:17.598652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:17.598652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:17.598652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:17.598652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:17.598652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:17.598652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:17.598652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:17.598652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:17.598652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:17.598652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:17.599655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:17.599655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:17.599655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:17.599655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:17.599655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:17.599655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:17.599655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:17.599655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:17.599655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:17.599655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:17.599655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:17.599655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:17.599655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:17.599655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:17.599655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:17.599655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:17.606007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:17.606007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:17.606007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:17.606007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:17.606007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:17.606007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:17.606007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:17.606007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:17.606007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:17.606007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:17.606007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:59:17.606007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:17.606007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 484/978] 10.222.0.93 () {56 vars in 1120 bytes} [Mon Aug 1 14:59:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:17.606762 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad77d5d-e69a-462e-bf2b-6d4523c4aa11 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:17.724788 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad77d5d-e69a-462e-bf2b-6d4523c4aa11 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:17.741474 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:17.768160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fced5-b4be-48c8-b641-ae19ac72d35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:17.768850 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fced5-b4be-48c8-b641-ae19ac72d35e None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:17.769234 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fced5-b4be-48c8-b641-ae19ac72d35e None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:17.769628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fced5-b4be-48c8-b641-ae19ac72d35e None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:17.770632 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fced5-b4be-48c8-b641-ae19ac72d35e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:17.770750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fced5-b4be-48c8-b641-ae19ac72d35e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:17.772205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fced5-b4be-48c8-b641-ae19ac72d35e None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:17.792511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 495/979] 10.222.0.93 () {56 vars in 1115 bytes} [Mon Aug 1 14:59:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:17.795380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d66898-0440-49c0-a733-dff2b347c8aa None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:17.910368 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:17.989028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d66898-0440-49c0-a733-dff2b347c8aa None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:18.042719 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b12c4-1e84-402d-a075-6ca5615ca176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:18.043716 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b12c4-1e84-402d-a075-6ca5615ca176 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:18.044205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b12c4-1e84-402d-a075-6ca5615ca176 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:18.044726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b12c4-1e84-402d-a075-6ca5615ca176 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:18.057163 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b12c4-1e84-402d-a075-6ca5615ca176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:18.058985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b12c4-1e84-402d-a075-6ca5615ca176 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:18.075955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-743b12c4-1e84-402d-a075-6ca5615ca176 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:18.075955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:18.075955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:18.075955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:18.075955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:18.075955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:18.075955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:18.075955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:18.075955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:18.075955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:18.075955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:18.075955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:18.075955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:59:18.075955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:59:18.075955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:59:18.077034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:59:18.077034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:18.077034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:18.077034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:18.077034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:18.077034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:18.077034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:18.077034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:18.077034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:18.077034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:18.077034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:18.077034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:18.077034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:18.077034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:18.077034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:18.077034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:18.077034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:18.078113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:59:18.078113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:59:18.078113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:18.078113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:18.086440 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 485/980] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:18.090940 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fced5-b4be-48c8-b641-ae19ac72d35e None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:18.229712 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fced5-b4be-48c8-b641-ae19ac72d35e None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:18.265911 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe87b94-abb6-433f-9d7f-09d1b7783a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:18.266532 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe87b94-abb6-433f-9d7f-09d1b7783a81 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:18.266812 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe87b94-abb6-433f-9d7f-09d1b7783a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:18.267086 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe87b94-abb6-433f-9d7f-09d1b7783a81 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:18.267772 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe87b94-abb6-433f-9d7f-09d1b7783a81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:18.268071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe87b94-abb6-433f-9d7f-09d1b7783a81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:18.270073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe87b94-abb6-433f-9d7f-09d1b7783a81 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:18.283027 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 496/981] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:18.289420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b12c4-1e84-402d-a075-6ca5615ca176 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:18.317888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706ced8-f442-4a60-bf10-a08d32643662 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:18.318303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706ced8-f442-4a60-bf10-a08d32643662 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:18.318842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706ced8-f442-4a60-bf10-a08d32643662 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:18.321508 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 497/982] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:18.329930 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926039e1-040c-4bba-8945-e814a395276c None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:18.330436 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926039e1-040c-4bba-8945-e814a395276c None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:18.330938 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926039e1-040c-4bba-8945-e814a395276c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:18.361366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-926039e1-040c-4bba-8945-e814a395276c None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:18.363774 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-926039e1-040c-4bba-8945-e814a395276c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:18.415030 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:18.422545 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b12c4-1e84-402d-a075-6ca5615ca176 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:18.435054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:18.463315 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 498/983] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:18.464813 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b4c96-a057-459f-ba71-b49398b40ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:18.465807 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b4c96-a057-459f-ba71-b49398b40ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:18.465807 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b4c96-a057-459f-ba71-b49398b40ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:18.466027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b4c96-a057-459f-ba71-b49398b40ee5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:18.475896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b4c96-a057-459f-ba71-b49398b40ee5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:18.478453 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b4c96-a057-459f-ba71-b49398b40ee5 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:18.489759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926039e1-040c-4bba-8945-e814a395276c None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:18.495355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-c06b4c96-a057-459f-ba71-b49398b40ee5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:59:18.495355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:18.495355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:18.495355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:18.495355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:18.495355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:18.495355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:18.495355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:18.495355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:18.495355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:18.495355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:18.495355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:18.495355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:59:18.495355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:59:18.495355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:59:18.495355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:59:18.496420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:18.496420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:18.496420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:59:18.496420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:59:18.496420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:59:18.496420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:59:18.496420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:18.496420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:18.496420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:18.496420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:18.496420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:18.496420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:18.496420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:18.496420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:18.496420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:18.496420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:18.496420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:18.496420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:18.497497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:18.497497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:18.497497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:59:18.497497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:59:18.497497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:59:18.497497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:59:18.497497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:59:18.497497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:59:18.497497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 14:59:18.497497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:18.497497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 486/984] 10.222.0.93 () {56 vars in 1117 bytes} [Mon Aug 1 14:59:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:18.501993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b4c96-a057-459f-ba71-b49398b40ee5 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:18.627269 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926039e1-040c-4bba-8945-e814a395276c None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:18.665153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b4c96-a057-459f-ba71-b49398b40ee5 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:18.671981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e38b7-b4d8-41f2-8448-981b497c62a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:18.672677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e38b7-b4d8-41f2-8448-981b497c62a6 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:18.672957 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e38b7-b4d8-41f2-8448-981b497c62a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:18.673228 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e38b7-b4d8-41f2-8448-981b497c62a6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:18.673879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e38b7-b4d8-41f2-8448-981b497c62a6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:18.675869 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e38b7-b4d8-41f2-8448-981b497c62a6 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:18.686013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-d13e38b7-b4d8-41f2-8448-981b497c62a6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 14:59:18.686013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:18.686013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:59:18.686013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:59:18.686013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:59:18.686013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:59:18.686013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:59:18.686013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:18.686013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 14:59:18.686013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:18.686013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:59:18.686013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:18.686013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:18.686013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:18.686013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:18.686013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:18.686013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:18.687613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:18.687613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:18.687613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:18.687613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:18.687613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:18.687613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:18.687613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:59:18.687613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:59:18.687613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:59:18.687613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:59:18.687613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:18.687613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:18.687613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:18.687613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:18.687613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:18.687613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:18.688814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:18.688814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:18.688814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:18.688814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:18.688814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:18.688814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:18.688814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:18.688814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:18.688814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:18.688814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:18.688814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:59:18.688814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:18.688814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 14:59:18.688814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:18.688814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 499/985] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:59:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:18.694284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e38b7-b4d8-41f2-8448-981b497c62a6 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:18.716238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3125372-2503-4164-8cb6-edac67a2f4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:18.717103 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3125372-2503-4164-8cb6-edac67a2f4cb None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:18.717642 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3125372-2503-4164-8cb6-edac67a2f4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:18.718209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3125372-2503-4164-8cb6-edac67a2f4cb None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:18.719053 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3125372-2503-4164-8cb6-edac67a2f4cb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:18.719375 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3125372-2503-4164-8cb6-edac67a2f4cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:18.720973 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3125372-2503-4164-8cb6-edac67a2f4cb None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:18.735152 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:18.746468 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 487/986] 10.222.0.93 () {56 vars in 1149 bytes} [Mon Aug 1 14:59:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:18.752304 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3125372-2503-4164-8cb6-edac67a2f4cb None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:18.791984 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:18.837688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e38b7-b4d8-41f2-8448-981b497c62a6 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:18.875714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84843f17-4948-4311-b1c2-26c95df99a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:18.876347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84843f17-4948-4311-b1c2-26c95df99a9e None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:18.876657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84843f17-4948-4311-b1c2-26c95df99a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:18.876934 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84843f17-4948-4311-b1c2-26c95df99a9e None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:18.877678 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84843f17-4948-4311-b1c2-26c95df99a9e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:18.877927 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84843f17-4948-4311-b1c2-26c95df99a9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:18.880344 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84843f17-4948-4311-b1c2-26c95df99a9e None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:18.883934 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3125372-2503-4164-8cb6-edac67a2f4cb None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:18.886123 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 500/987] 10.222.0.93 () {56 vars in 1112 bytes} [Mon Aug 1 14:59:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:18.887276 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb9d2f-fe5b-40e5-9e9c-16a4fb822f2c None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:18.887545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb9d2f-fe5b-40e5-9e9c-16a4fb822f2c None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:18.887815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb9d2f-fe5b-40e5-9e9c-16a4fb822f2c None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:18.889188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 501/988] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:18.894140 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb9d2f-fe5b-40e5-9e9c-16a4fb822f2c None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:18.927565 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538aa5ef-1a1d-4733-af7e-5b16d456362b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:18.928202 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538aa5ef-1a1d-4733-af7e-5b16d456362b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:18.928501 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538aa5ef-1a1d-4733-af7e-5b16d456362b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:18.928766 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538aa5ef-1a1d-4733-af7e-5b16d456362b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:18.998592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:19.009227 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538aa5ef-1a1d-4733-af7e-5b16d456362b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:19.010691 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538aa5ef-1a1d-4733-af7e-5b16d456362b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:19.015783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb9d2f-fe5b-40e5-9e9c-16a4fb822f2c None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:19.042464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:19.064801 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e3c6e-df80-4eb1-bf27-79567735a43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:19.065432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e3c6e-df80-4eb1-bf27-79567735a43f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:19.065713 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e3c6e-df80-4eb1-bf27-79567735a43f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:19.066034 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e3c6e-df80-4eb1-bf27-79567735a43f None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:19.066740 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e3c6e-df80-4eb1-bf27-79567735a43f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:19.067042 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e3c6e-df80-4eb1-bf27-79567735a43f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:19.068437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e3c6e-df80-4eb1-bf27-79567735a43f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:19.073740 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 502/989] 10.222.0.93 () {56 vars in 1112 bytes} [Mon Aug 1 14:59:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:19.075065 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39594e97-8eec-41ab-8dbf-b2141dbbc59a None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:19.075343 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39594e97-8eec-41ab-8dbf-b2141dbbc59a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:19.075615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39594e97-8eec-41ab-8dbf-b2141dbbc59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:19.094778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-39594e97-8eec-41ab-8dbf-b2141dbbc59a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:19.096013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-39594e97-8eec-41ab-8dbf-b2141dbbc59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:19.137317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 488/990] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:59:18 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 14:59:19.138435 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538aa5ef-1a1d-4733-af7e-5b16d456362b None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:19.181207 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 503/991] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:19.182245 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39594e97-8eec-41ab-8dbf-b2141dbbc59a None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:19.275886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538aa5ef-1a1d-4733-af7e-5b16d456362b None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:19.287565 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:19.306229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39594e97-8eec-41ab-8dbf-b2141dbbc59a None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:19.316757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:19.328364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a618c-8dbe-4043-b208-17e42d88546d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:19.329389 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a618c-8dbe-4043-b208-17e42d88546d None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:19.329880 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a618c-8dbe-4043-b208-17e42d88546d None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:19.330427 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a618c-8dbe-4043-b208-17e42d88546d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:19.331714 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a618c-8dbe-4043-b208-17e42d88546d None admin] RBAC: Adding query filter params (interface=public, service_id=2a19f248c25649578433f6876a9623d0, region_id=RegionOne) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:19.332227 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a618c-8dbe-4043-b208-17e42d88546d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:19.334685 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a618c-8dbe-4043-b208-17e42d88546d None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:19.348525 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 489/992] 10.222.0.93 () {56 vars in 1251 bytes} [Mon Aug 1 14:59:19 2022] GET /identity/v3/endpoints?service_id=2a19f248c25649578433f6876a9623d0&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:19.348525 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a618c-8dbe-4043-b208-17e42d88546d None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:19.357336 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262ebb1-6f08-4c33-b14c-34416dde6829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:19.358337 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262ebb1-6f08-4c33-b14c-34416dde6829 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:19.358782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262ebb1-6f08-4c33-b14c-34416dde6829 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:19.359237 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262ebb1-6f08-4c33-b14c-34416dde6829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:19.373071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262ebb1-6f08-4c33-b14c-34416dde6829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:19.374545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262ebb1-6f08-4c33-b14c-34416dde6829 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:19.395733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-b262ebb1-6f08-4c33-b14c-34416dde6829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:19.395733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:19.395733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:19.395733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:19.395733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:19.395733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:19.395733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:19.395733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:19.395733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:19.395733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:19.395733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:19.395733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:19.395733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:59:19.395733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:59:19.395733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:59:19.397605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:59:19.397605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:19.397605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:19.397605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:19.397605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:19.397605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:19.397605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:19.397605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:19.397605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:19.397605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:19.397605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:19.397605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:19.397605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:19.397605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:19.397605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:19.397605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:19.397605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:19.399562 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:59:19.399562 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:59:19.399562 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:19.399562 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:19.399562 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 504/993] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:19.399562 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262ebb1-6f08-4c33-b14c-34416dde6829 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:19.486794 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a618c-8dbe-4043-b208-17e42d88546d None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:19.529768 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78dc8e4-3441-4758-afd2-ac621a08e4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:19.530421 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78dc8e4-3441-4758-afd2-ac621a08e4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:19.530700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78dc8e4-3441-4758-afd2-ac621a08e4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:19.531177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78dc8e4-3441-4758-afd2-ac621a08e4e9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:19.531711 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78dc8e4-3441-4758-afd2-ac621a08e4e9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:19.533168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78dc8e4-3441-4758-afd2-ac621a08e4e9 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:19.543894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-c78dc8e4-3441-4758-afd2-ac621a08e4e9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:59:19.543894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:19.543894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:59:19.543894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:59:19.543894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:59:19.543894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:59:19.543894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:59:19.543894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:19.543894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:59:19.543894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:19.543894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:59:19.543894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:19.543894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:19.543894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:19.543894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:19.543894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:19.543894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:19.545772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:19.545772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:19.545772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:19.545772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:19.545772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:19.545772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:19.545772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:59:19.545772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:59:19.545772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:59:19.545772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:59:19.545772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:19.545772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:19.545772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:19.545772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:19.545772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:19.545772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:19.547497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:19.547497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:19.547497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:19.547497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:19.547497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:19.547497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:19.547497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:19.547497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:19.547497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:19.547497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:19.547497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:59:19.547497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:19.547497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 14:59:19.547497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:19.547497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 490/994] 10.222.0.93 () {56 vars in 1126 bytes} [Mon Aug 1 14:59:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:19.549061 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78dc8e4-3441-4758-afd2-ac621a08e4e9 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:19.554804 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262ebb1-6f08-4c33-b14c-34416dde6829 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:19.599374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c2933-3268-44a0-82cc-e51c6110b5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:19.600325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c2933-3268-44a0-82cc-e51c6110b5dc None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:19.600799 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c2933-3268-44a0-82cc-e51c6110b5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:19.601232 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c2933-3268-44a0-82cc-e51c6110b5dc None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:19.602850 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449c2933-3268-44a0-82cc-e51c6110b5dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:19.604394 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449c2933-3268-44a0-82cc-e51c6110b5dc None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:19.612718 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 505/995] 10.222.0.93 () {56 vars in 1087 bytes} [Mon Aug 1 14:59:19 2022] GET /identity/v3/services => generated 1727 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:59:19.613767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c2933-3268-44a0-82cc-e51c6110b5dc None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:19.616950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:19.629486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:19.703210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78dc8e4-3441-4758-afd2-ac621a08e4e9 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:19.765731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b48f0a-efff-4d4f-b892-b732f6f4e2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:19.766577 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b48f0a-efff-4d4f-b892-b732f6f4e2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:19.766988 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b48f0a-efff-4d4f-b892-b732f6f4e2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:19.767364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b48f0a-efff-4d4f-b892-b732f6f4e2a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:19.768200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b48f0a-efff-4d4f-b892-b732f6f4e2a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:19.768835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b48f0a-efff-4d4f-b892-b732f6f4e2a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:19.771921 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b48f0a-efff-4d4f-b892-b732f6f4e2a8 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:19.788420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 491/996] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:19.792547 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c2933-3268-44a0-82cc-e51c6110b5dc None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:19.796204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b48f0a-efff-4d4f-b892-b732f6f4e2a8 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:19.851165 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df037d-9266-4a7b-9ef4-108e6fe7581b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:19.852039 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df037d-9266-4a7b-9ef4-108e6fe7581b None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:19.854246 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df037d-9266-4a7b-9ef4-108e6fe7581b None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:19.854246 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df037d-9266-4a7b-9ef4-108e6fe7581b None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:19.854246 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df037d-9266-4a7b-9ef4-108e6fe7581b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:19.854737 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df037d-9266-4a7b-9ef4-108e6fe7581b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:19.857127 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df037d-9266-4a7b-9ef4-108e6fe7581b None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:19.870320 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 506/997] 10.222.0.93 () {56 vars in 1122 bytes} [Mon Aug 1 14:59:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:19.881182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df037d-9266-4a7b-9ef4-108e6fe7581b None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:19.964612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b48f0a-efff-4d4f-b892-b732f6f4e2a8 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:19.969584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:19.996165 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:20.025952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df037d-9266-4a7b-9ef4-108e6fe7581b None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:20.030472 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec075b-e265-4a34-a7e7-b70dd750a1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:20.031908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec075b-e265-4a34-a7e7-b70dd750a1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:20.031908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec075b-e265-4a34-a7e7-b70dd750a1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:20.032150 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec075b-e265-4a34-a7e7-b70dd750a1e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:20.047153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec075b-e265-4a34-a7e7-b70dd750a1e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:20.049529 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec075b-e265-4a34-a7e7-b70dd750a1e5 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:20.069913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-4aec075b-e265-4a34-a7e7-b70dd750a1e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:59:20.069913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:20.069913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:20.069913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:20.069913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:20.069913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:20.069913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:20.069913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:20.069913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:20.069913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:20.069913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:20.069913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:20.069913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:59:20.069913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:59:20.069913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:59:20.077506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:59:20.077506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:20.077506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:20.077506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:20.077506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:20.077506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:20.077506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:20.077506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:20.077506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:20.077506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:20.077506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:20.077506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:20.077506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:20.077506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:20.077506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:20.077506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:20.077506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:20.082640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:59:20.082640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:59:20.082640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:59:20.082640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:59:20.082640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:59:20.082640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:59:20.082640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:59:20.082640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:20.082640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 492/998] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:59:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:20.085301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec075b-e265-4a34-a7e7-b70dd750a1e5 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:20.095153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b9070-1d96-4da6-b58d-c98ad2fe912d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:20.095783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b9070-1d96-4da6-b58d-c98ad2fe912d None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:20.096057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b9070-1d96-4da6-b58d-c98ad2fe912d None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:20.096328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b9070-1d96-4da6-b58d-c98ad2fe912d None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:20.097097 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b9070-1d96-4da6-b58d-c98ad2fe912d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:20.097400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b9070-1d96-4da6-b58d-c98ad2fe912d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:20.107304 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b9070-1d96-4da6-b58d-c98ad2fe912d None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:20.107304 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 507/999] 10.222.0.93 () {56 vars in 1122 bytes} [Mon Aug 1 14:59:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:20.114638 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b9070-1d96-4da6-b58d-c98ad2fe912d None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:20.261614 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec075b-e265-4a34-a7e7-b70dd750a1e5 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:20.294029 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b9070-1d96-4da6-b58d-c98ad2fe912d None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:20.322254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ce9c3-cb6b-4e7e-a491-abaf37aa81ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:20.323256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ce9c3-cb6b-4e7e-a491-abaf37aa81ba None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:20.323704 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ce9c3-cb6b-4e7e-a491-abaf37aa81ba None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:20.324070 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ce9c3-cb6b-4e7e-a491-abaf37aa81ba None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:20.325109 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ce9c3-cb6b-4e7e-a491-abaf37aa81ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:20.325417 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ce9c3-cb6b-4e7e-a491-abaf37aa81ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:20.327120 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ce9c3-cb6b-4e7e-a491-abaf37aa81ba None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:20.342324 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:20.348534 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 493/1000] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:59:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:20.354755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7c544-804e-4bce-a1bd-49319e75ba41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:20.355825 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7c544-804e-4bce-a1bd-49319e75ba41 None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:20.356286 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7c544-804e-4bce-a1bd-49319e75ba41 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:20.356790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7c544-804e-4bce-a1bd-49319e75ba41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:20.357440 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ce9c3-cb6b-4e7e-a491-abaf37aa81ba None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:20.358144 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7c544-804e-4bce-a1bd-49319e75ba41 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:20.360628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7c544-804e-4bce-a1bd-49319e75ba41 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:20.388297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 508/1001] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 14:59:20 2022] POST /identity/v3/endpoints => generated 325 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:59:20.442242 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:20.576369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ce9c3-cb6b-4e7e-a491-abaf37aa81ba None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:20.629618 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0d9b3-0dcb-47fb-96aa-ada931d6fb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:20.629618 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0d9b3-0dcb-47fb-96aa-ada931d6fb10 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:20.629998 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0d9b3-0dcb-47fb-96aa-ada931d6fb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:20.630307 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0d9b3-0dcb-47fb-96aa-ada931d6fb10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:20.630972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0d9b3-0dcb-47fb-96aa-ada931d6fb10 None admin] RBAC: Adding query filter params (role.id=66135f6413e94a77b7c0762b52a4d38e, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=4370f8f474c2446e989d0d8c5b75e87a) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:20.631280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0d9b3-0dcb-47fb-96aa-ada931d6fb10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:20.632742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0d9b3-0dcb-47fb-96aa-ada931d6fb10 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:20.640336 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 494/1002] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:59:20 2022] GET /identity/v3/role_assignments?user.id=4370f8f474c2446e989d0d8c5b75e87a&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=66135f6413e94a77b7c0762b52a4d38e => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:22.473695 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1ca7a-61b2-41ff-866d-13ed054b098f None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:22.474524 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1ca7a-61b2-41ff-866d-13ed054b098f None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:22.474700 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1ca7a-61b2-41ff-866d-13ed054b098f None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:22.477274 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 509/1003] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:22.483347 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb8adf-250c-47c0-a466-41329acb9a22 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:22.484086 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb8adf-250c-47c0-a466-41329acb9a22 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:22.484086 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb8adf-250c-47c0-a466-41329acb9a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:22.508344 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-97eb8adf-250c-47c0-a466-41329acb9a22 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:22.509829 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-97eb8adf-250c-47c0-a466-41329acb9a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:22.605093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 495/1004] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:22 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:22.630315 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1ca7a-61b2-41ff-866d-13ed054b098f None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:22.725761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d558e33-9762-4d3d-af7e-e3e3a1a9f643 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:22.725999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d558e33-9762-4d3d-af7e-e3e3a1a9f643 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:22.726346 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d558e33-9762-4d3d-af7e-e3e3a1a9f643 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:22.727983 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 496/1005] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:22.733924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c62b1-d49c-424a-a5f5-02a587979e9f None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:22.734222 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c62b1-d49c-424a-a5f5-02a587979e9f None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:22.734480 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c62b1-d49c-424a-a5f5-02a587979e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:22.759660 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-466c62b1-d49c-424a-a5f5-02a587979e9f None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:22.761380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-466c62b1-d49c-424a-a5f5-02a587979e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:22.764210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1ca7a-61b2-41ff-866d-13ed054b098f None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:22.791876 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:22.817250 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc9891-6c52-4868-8415-774cb9035a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:22.818321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc9891-6c52-4868-8415-774cb9035a7a None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:22.818692 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc9891-6c52-4868-8415-774cb9035a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:22.819139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc9891-6c52-4868-8415-774cb9035a7a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:22.820174 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc9891-6c52-4868-8415-774cb9035a7a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:22.822469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc9891-6c52-4868-8415-774cb9035a7a None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:22.837143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-c0cc9891-6c52-4868-8415-774cb9035a7a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 14:59:22.837143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:22.837143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:59:22.837143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:59:22.837143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:59:22.837143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:59:22.837143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:59:22.837143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:22.837143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 14:59:22.837143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:22.837143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:59:22.837143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:22.837143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:22.837143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:22.837143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:22.837143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:22.837143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:22.839159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:22.839159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:22.839159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:22.839159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:22.839159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:22.839159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:22.839159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:59:22.839159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:59:22.839159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:59:22.839159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:59:22.839159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:22.839159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:22.839159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:22.839159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:22.839159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:22.839159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:22.840696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:22.840696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:22.840696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:22.840696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:22.840696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:22.840696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:22.840696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:22.840696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:22.840696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:22.840696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:22.840696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:59:22.840696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:22.840696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 14:59:22.840696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:22.840696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 510/1006] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:59:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:22.846651 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc9891-6c52-4868-8415-774cb9035a7a None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:22.882152 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 497/1007] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:22 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:59:22.910303 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c62b1-d49c-424a-a5f5-02a587979e9f None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:22.935634 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:23.017159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc9891-6c52-4868-8415-774cb9035a7a None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:23.055477 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c62b1-d49c-424a-a5f5-02a587979e9f None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:23.074818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361ca15-de71-4585-a92b-35c9def00e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:23.075747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361ca15-de71-4585-a92b-35c9def00e6c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:23.076155 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361ca15-de71-4585-a92b-35c9def00e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:23.076676 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361ca15-de71-4585-a92b-35c9def00e6c None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:23.077927 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361ca15-de71-4585-a92b-35c9def00e6c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:23.078471 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361ca15-de71-4585-a92b-35c9def00e6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:23.081108 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361ca15-de71-4585-a92b-35c9def00e6c None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:23.091217 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 511/1008] 10.222.0.93 () {56 vars in 1112 bytes} [Mon Aug 1 14:59:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:23.099802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361ca15-de71-4585-a92b-35c9def00e6c None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:23.112260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee711e7-9719-460b-9816-234754f515d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:23.113925 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee711e7-9719-460b-9816-234754f515d2 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:23.114331 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee711e7-9719-460b-9816-234754f515d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:23.115435 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee711e7-9719-460b-9816-234754f515d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:23.117645 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee711e7-9719-460b-9816-234754f515d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:23.121434 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee711e7-9719-460b-9816-234754f515d2 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:23.128754 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 498/1009] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:23.134102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee711e7-9719-460b-9816-234754f515d2 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:23.181814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:23.277091 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361ca15-de71-4585-a92b-35c9def00e6c None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:23.296667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:23.302884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee711e7-9719-460b-9816-234754f515d2 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:23.317953 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f56c86-450b-4dc4-8d4b-0c8c20da921c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:23.318614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f56c86-450b-4dc4-8d4b-0c8c20da921c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:23.318954 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f56c86-450b-4dc4-8d4b-0c8c20da921c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:23.319233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f56c86-450b-4dc4-8d4b-0c8c20da921c None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:23.319909 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f56c86-450b-4dc4-8d4b-0c8c20da921c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:23.320204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f56c86-450b-4dc4-8d4b-0c8c20da921c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:23.321623 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f56c86-450b-4dc4-8d4b-0c8c20da921c None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:23.326833 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 512/1010] 10.222.0.93 () {56 vars in 1112 bytes} [Mon Aug 1 14:59:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:23.328154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28bb13-3247-4c09-9f40-4a4f413b7536 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:23.328413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28bb13-3247-4c09-9f40-4a4f413b7536 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:23.328719 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28bb13-3247-4c09-9f40-4a4f413b7536 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:23.332513 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 513/1011] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:23.334749 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28bb13-3247-4c09-9f40-4a4f413b7536 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:23.355534 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0fe34-5b06-4bfa-8caa-7d4d657b3069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:23.356455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0fe34-5b06-4bfa-8caa-7d4d657b3069 None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:23.356946 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0fe34-5b06-4bfa-8caa-7d4d657b3069 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:23.357387 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0fe34-5b06-4bfa-8caa-7d4d657b3069 None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:23.358522 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0fe34-5b06-4bfa-8caa-7d4d657b3069 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:23.360362 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0fe34-5b06-4bfa-8caa-7d4d657b3069 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:23.397464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 499/1012] 10.222.0.93 () {60 vars in 1141 bytes} [Mon Aug 1 14:59:23 2022] POST /identity/v3/projects => generated 311 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:59:23.398672 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60d847-ced8-419a-8769-10e0f8517e69 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:23.398963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60d847-ced8-419a-8769-10e0f8517e69 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:23.399241 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60d847-ced8-419a-8769-10e0f8517e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:23.433335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-af60d847-ced8-419a-8769-10e0f8517e69 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:23.435306 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-af60d847-ced8-419a-8769-10e0f8517e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:23.506310 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28bb13-3247-4c09-9f40-4a4f413b7536 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:23.554282 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 500/1013] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:23 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:23.556368 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8257bdc-e860-48cf-87cf-06d2a4ee48d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:23.557046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8257bdc-e860-48cf-87cf-06d2a4ee48d2 None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:23.557379 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8257bdc-e860-48cf-87cf-06d2a4ee48d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:23.557657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8257bdc-e860-48cf-87cf-06d2a4ee48d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:23.558380 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8257bdc-e860-48cf-87cf-06d2a4ee48d2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:23.559766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8257bdc-e860-48cf-87cf-06d2a4ee48d2 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:23.578024 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60d847-ced8-419a-8769-10e0f8517e69 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:23.587447 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:23.589820 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 514/1014] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 14:59:23 2022] POST /identity/v3/endpoints => generated 334 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:59:23.763804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60d847-ced8-419a-8769-10e0f8517e69 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:23.838780 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43492479-6634-43f2-be1c-bdcd21e91cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:23.839449 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43492479-6634-43f2-be1c-bdcd21e91cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:23.839724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43492479-6634-43f2-be1c-bdcd21e91cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:23.840002 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43492479-6634-43f2-be1c-bdcd21e91cb2 None admin] PATH_INFO: `/v3/services/image` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:23.840659 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43492479-6634-43f2-be1c-bdcd21e91cb2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:23.842208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43492479-6634-43f2-be1c-bdcd21e91cb2 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:23.853393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-43492479-6634-43f2-be1c-bdcd21e91cb2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 14:59:23.853393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:23.853393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:59:23.853393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:59:23.853393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:59:23.853393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:59:23.853393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:59:23.853393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:23.853393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 14:59:23.853393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:23.853393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:59:23.853393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:23.853393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:23.853393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:23.853393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:23.853393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:23.853393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:23.855023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:23.855023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:23.855023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:23.855023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:23.855023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:23.855023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:23.855023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:59:23.855023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:59:23.855023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:59:23.855023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:59:23.855023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:23.855023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:23.855023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:23.855023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:23.855023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:23.855023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:23.856205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:23.856205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:23.856205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:23.856205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:23.856205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:23.856205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:23.856205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:23.856205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:23.856205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:23.856205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:23.856205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:59:23.856205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:23.856205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 14:59:23.856205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:23.856205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 501/1015] 10.222.0.93 () {56 vars in 1105 bytes} [Mon Aug 1 14:59:23 2022] GET /identity/v3/services/image => generated 86 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:23.865408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8257bdc-e860-48cf-87cf-06d2a4ee48d2 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:23.926515 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:24.024349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8257bdc-e860-48cf-87cf-06d2a4ee48d2 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:24.073858 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2f158-be6f-4f03-8364-4c34fee99e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:24.076997 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2f158-be6f-4f03-8364-4c34fee99e38 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:24.076997 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2f158-be6f-4f03-8364-4c34fee99e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:24.076997 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2f158-be6f-4f03-8364-4c34fee99e38 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:24.076997 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2f158-be6f-4f03-8364-4c34fee99e38 None admin] RBAC: Adding query filter params (name=image) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:24.076997 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2f158-be6f-4f03-8364-4c34fee99e38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:24.078311 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2f158-be6f-4f03-8364-4c34fee99e38 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:24.085437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 515/1016] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:24.093794 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43492479-6634-43f2-be1c-bdcd21e91cb2 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:24.228076 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:24.308531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43492479-6634-43f2-be1c-bdcd21e91cb2 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:24.380355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4ab8a-3aae-45a2-befe-b2e097539f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:24.381473 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4ab8a-3aae-45a2-befe-b2e097539f0b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:24.381973 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4ab8a-3aae-45a2-befe-b2e097539f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:24.382470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4ab8a-3aae-45a2-befe-b2e097539f0b None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:24.383921 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4ab8a-3aae-45a2-befe-b2e097539f0b None admin] RBAC: Adding query filter params (type=image) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:24.384489 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4ab8a-3aae-45a2-befe-b2e097539f0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:24.387349 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4ab8a-3aae-45a2-befe-b2e097539f0b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:24.400583 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 502/1017] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:24 2022] GET /identity/v3/services?type=image => generated 122 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:25.906588 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23b8ab-543f-414f-b053-1803535d2983 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:25.907419 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23b8ab-543f-414f-b053-1803535d2983 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:25.907536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23b8ab-543f-414f-b053-1803535d2983 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:25.910787 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 516/1018] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:25.918765 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48e877-05ad-423b-8cc2-583a7dba926f None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:25.919537 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48e877-05ad-423b-8cc2-583a7dba926f None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:25.919727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48e877-05ad-423b-8cc2-583a7dba926f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:25.945889 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-6e48e877-05ad-423b-8cc2-583a7dba926f None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:25.947213 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-6e48e877-05ad-423b-8cc2-583a7dba926f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:26.072570 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 503/1019] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:25 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:59:26.115781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23b8ab-543f-414f-b053-1803535d2983 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:26.318500 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23b8ab-543f-414f-b053-1803535d2983 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:26.321167 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:26.335536 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6a801-71a9-473d-a368-449201dcb874 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:26.336002 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6a801-71a9-473d-a368-449201dcb874 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:26.336370 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6a801-71a9-473d-a368-449201dcb874 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:26.338965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 504/1020] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:59:26.345187 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f8f9d-b1ff-4d5b-a601-d07223fa6fea None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:26.345541 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f8f9d-b1ff-4d5b-a601-d07223fa6fea None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:26.345942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f8f9d-b1ff-4d5b-a601-d07223fa6fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:26.374418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-e40f8f9d-b1ff-4d5b-a601-d07223fa6fea None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:26.376364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e40f8f9d-b1ff-4d5b-a601-d07223fa6fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:26.392612 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8c850-36c2-40bb-8060-a337daa22d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:26.394186 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8c850-36c2-40bb-8060-a337daa22d87 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:26.394655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8c850-36c2-40bb-8060-a337daa22d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:26.395428 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8c850-36c2-40bb-8060-a337daa22d87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:26.396934 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8c850-36c2-40bb-8060-a337daa22d87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:26.399252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8c850-36c2-40bb-8060-a337daa22d87 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:26.404544 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 517/1021] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:26.408105 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8c850-36c2-40bb-8060-a337daa22d87 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:26.481958 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:26.504739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 505/1022] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:26 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:59:26.541545 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f8f9d-b1ff-4d5b-a601-d07223fa6fea None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:26.582225 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8c850-36c2-40bb-8060-a337daa22d87 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:26.625011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a8da3-9dea-4024-9b93-d5aeda05dbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:26.625677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a8da3-9dea-4024-9b93-d5aeda05dbf9 None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:26.625993 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a8da3-9dea-4024-9b93-d5aeda05dbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:26.626371 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a8da3-9dea-4024-9b93-d5aeda05dbf9 None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:26.627075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a8da3-9dea-4024-9b93-d5aeda05dbf9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:26.629754 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a8da3-9dea-4024-9b93-d5aeda05dbf9 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:26.661544 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 518/1023] 10.222.0.93 () {60 vars in 1132 bytes} [Mon Aug 1 14:59:26 2022] POST /identity/v3/users => generated 285 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:59:26.663721 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b93ac3-e4ee-458f-966b-a0270926e99a None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:26.664188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b93ac3-e4ee-458f-966b-a0270926e99a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:26.664753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b93ac3-e4ee-458f-966b-a0270926e99a None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:26.667487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 519/1024] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:26.677366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb10150-701f-4539-8781-2f3e80a1f6e0 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:26.677844 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb10150-701f-4539-8781-2f3e80a1f6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:26.678302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb10150-701f-4539-8781-2f3e80a1f6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:26.710244 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-7bb10150-701f-4539-8781-2f3e80a1f6e0 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:26.711800 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-7bb10150-701f-4539-8781-2f3e80a1f6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:26.722364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f8f9d-b1ff-4d5b-a601-d07223fa6fea None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:26.739759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:26.784565 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7853f-78dd-4f19-b4d6-0e651665a233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:26.785287 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7853f-78dd-4f19-b4d6-0e651665a233 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:26.785805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7853f-78dd-4f19-b4d6-0e651665a233 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:26.786354 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7853f-78dd-4f19-b4d6-0e651665a233 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:26.804978 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7853f-78dd-4f19-b4d6-0e651665a233 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:26.813795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-63b7853f-78dd-4f19-b4d6-0e651665a233 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:59:26.813795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:26.813795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:26.813795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:26.813795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:26.813795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:26.813795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:26.813795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:26.813795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:26.813795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:26.813795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:26.813795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:26.813795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:26.813795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:26.813795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:26.813795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:26.815004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:26.815004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:26.815004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:26.815004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:26.815004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:26.815004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:26.815004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:26.815004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:26.815004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:26.815004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:26.815004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:26.815004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:26.815004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:26.815004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:26.815004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:26.815004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:26.817681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:26.817681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:26.817681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:26.817681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:26.817681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:26.817681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:26.817681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:26.817681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:26.817681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:26.817681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:26.817681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:59:26.817681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:26.817681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 506/1025] 10.222.0.93 () {56 vars in 1120 bytes} [Mon Aug 1 14:59:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:26.825425 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7853f-78dd-4f19-b4d6-0e651665a233 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:26.852257 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 520/1026] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:26 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:59:26.877243 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb10150-701f-4539-8781-2f3e80a1f6e0 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:26.938607 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:27.042110 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7853f-78dd-4f19-b4d6-0e651665a233 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:27.060369 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb10150-701f-4539-8781-2f3e80a1f6e0 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:27.088233 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325e766-fc26-44a8-abb7-493daf5858dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:27.088985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325e766-fc26-44a8-abb7-493daf5858dc None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:27.089339 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325e766-fc26-44a8-abb7-493daf5858dc None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:27.089686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325e766-fc26-44a8-abb7-493daf5858dc None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:27.090494 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325e766-fc26-44a8-abb7-493daf5858dc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:27.090801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325e766-fc26-44a8-abb7-493daf5858dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:27.092216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325e766-fc26-44a8-abb7-493daf5858dc None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:27.107856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 507/1027] 10.222.0.93 () {56 vars in 1115 bytes} [Mon Aug 1 14:59:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:27.110879 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325e766-fc26-44a8-abb7-493daf5858dc None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:27.122221 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be1999-dd66-4129-8470-8d909af6423a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:27.122954 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be1999-dd66-4129-8470-8d909af6423a None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:27.123462 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be1999-dd66-4129-8470-8d909af6423a None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:27.123677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be1999-dd66-4129-8470-8d909af6423a None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:27.124574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be1999-dd66-4129-8470-8d909af6423a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:27.126060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be1999-dd66-4129-8470-8d909af6423a None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:27.148174 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 521/1028] 10.222.0.93 () {60 vars in 1141 bytes} [Mon Aug 1 14:59:26 2022] POST /identity/v3/services => generated 242 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:59:27.261992 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325e766-fc26-44a8-abb7-493daf5858dc None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:27.278920 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:27.325939 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71225e-a121-42aa-a150-377bd7400138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:27.326831 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71225e-a121-42aa-a150-377bd7400138 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:27.327244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71225e-a121-42aa-a150-377bd7400138 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:27.327656 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71225e-a121-42aa-a150-377bd7400138 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:27.343876 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71225e-a121-42aa-a150-377bd7400138 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:27.345488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71225e-a121-42aa-a150-377bd7400138 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:27.363625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-4a71225e-a121-42aa-a150-377bd7400138 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:27.363625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:27.363625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:27.363625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:27.363625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:27.363625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:27.363625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:27.363625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:27.363625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:27.363625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:27.363625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:27.363625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:27.363625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:59:27.363625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:59:27.363625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:59:27.366040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:59:27.366040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:27.366040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:27.366040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:27.366040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:27.366040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:27.366040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:27.366040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:27.366040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:27.366040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:27.366040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:27.366040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:27.366040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:27.366040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:27.366040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:27.366040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:27.366040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:27.367758 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:59:27.367758 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:59:27.367758 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:27.367758 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:27.367758 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 508/1029] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:27.373359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be1999-dd66-4129-8470-8d909af6423a None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:27.399204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:27.522897 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be1999-dd66-4129-8470-8d909af6423a None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:27.565306 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d286d3a-a6be-4ba7-9573-20c699594e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:27.566131 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d286d3a-a6be-4ba7-9573-20c699594e26 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:27.566542 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d286d3a-a6be-4ba7-9573-20c699594e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:27.566949 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d286d3a-a6be-4ba7-9573-20c699594e26 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:27.567834 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d286d3a-a6be-4ba7-9573-20c699594e26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:27.568311 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d286d3a-a6be-4ba7-9573-20c699594e26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:27.569915 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d286d3a-a6be-4ba7-9573-20c699594e26 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:27.589717 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 522/1030] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:27.595873 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71225e-a121-42aa-a150-377bd7400138 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:27.725311 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71225e-a121-42aa-a150-377bd7400138 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:27.774463 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354f11a-ec65-4775-98f3-bc862e773bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:27.775590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354f11a-ec65-4775-98f3-bc862e773bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:27.776101 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354f11a-ec65-4775-98f3-bc862e773bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:27.776650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354f11a-ec65-4775-98f3-bc862e773bb3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:27.791698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354f11a-ec65-4775-98f3-bc862e773bb3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:27.793223 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354f11a-ec65-4775-98f3-bc862e773bb3 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:27.808153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-c354f11a-ec65-4775-98f3-bc862e773bb3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:59:27.808153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:27.808153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:27.808153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:27.808153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:27.808153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:27.808153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:27.808153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:27.808153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:27.808153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:27.808153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:27.808153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:27.808153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:59:27.808153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:59:27.808153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:59:27.808153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:59:27.809996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:27.809996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:27.809996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:59:27.809996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:59:27.809996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:59:27.809996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:59:27.809996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:27.809996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:27.809996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:27.809996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:27.809996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:27.809996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:27.809996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:27.809996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:27.809996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:27.809996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:27.809996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:27.809996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:27.813081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:27.813081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:27.813081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:59:27.813081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:59:27.813081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:59:27.813081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:59:27.813081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:59:27.813081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:59:27.813081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:59:27.813081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:27.818613 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 509/1031] 10.222.0.93 () {56 vars in 1093 bytes} [Mon Aug 1 14:59:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:27.819461 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d286d3a-a6be-4ba7-9573-20c699594e26 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:27.969267 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:28.015143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d286d3a-a6be-4ba7-9573-20c699594e26 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:28.078202 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d0104-e3c8-4305-8031-c3b40159cf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:28.079260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d0104-e3c8-4305-8031-c3b40159cf37 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:28.079769 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d0104-e3c8-4305-8031-c3b40159cf37 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:28.080262 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d0104-e3c8-4305-8031-c3b40159cf37 None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:28.082426 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d0104-e3c8-4305-8031-c3b40159cf37 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:28.083000 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d0104-e3c8-4305-8031-c3b40159cf37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:28.085861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d0104-e3c8-4305-8031-c3b40159cf37 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:28.124217 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 523/1032] 10.222.0.93 () {56 vars in 1133 bytes} [Mon Aug 1 14:59:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:28.126542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354f11a-ec65-4775-98f3-bc862e773bb3 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:28.165470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:28.279757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354f11a-ec65-4775-98f3-bc862e773bb3 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:28.338419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd835424-0dea-4ea9-9ebc-26590fc8ba5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:28.339472 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd835424-0dea-4ea9-9ebc-26590fc8ba5d None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:28.340012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd835424-0dea-4ea9-9ebc-26590fc8ba5d None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:28.340587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd835424-0dea-4ea9-9ebc-26590fc8ba5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:28.462581 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd835424-0dea-4ea9-9ebc-26590fc8ba5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:28.464676 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd835424-0dea-4ea9-9ebc-26590fc8ba5d None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:28.501820 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:28.631188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 510/1033] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:59:28 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 14:59:28.636950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d0104-e3c8-4305-8031-c3b40159cf37 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:28.766219 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d0104-e3c8-4305-8031-c3b40159cf37 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:28.818713 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe992f0-91ee-4ae9-b167-265860a6c9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:28.819370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe992f0-91ee-4ae9-b167-265860a6c9fd None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:28.819644 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe992f0-91ee-4ae9-b167-265860a6c9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:28.819935 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe992f0-91ee-4ae9-b167-265860a6c9fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:28.822302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:28.833529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe992f0-91ee-4ae9-b167-265860a6c9fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:28.835004 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe992f0-91ee-4ae9-b167-265860a6c9fd None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:28.851791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-3fe992f0-91ee-4ae9-b167-265860a6c9fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:28.851791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:28.851791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:28.851791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:28.851791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:28.851791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:28.851791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:28.851791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:28.851791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:28.851791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:28.851791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:28.851791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:28.851791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:59:28.851791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:59:28.851791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:59:28.853376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:59:28.853376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:28.853376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:28.853376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:28.853376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:28.853376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:28.853376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:28.853376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:28.853376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:28.853376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:28.853376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:28.853376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:28.853376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:28.853376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:28.853376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:28.853376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:28.853376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:28.854992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:59:28.854992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:59:28.854992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:28.854992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:28.856869 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 524/1034] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:28.859122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd835424-0dea-4ea9-9ebc-26590fc8ba5d None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:29.012335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd835424-0dea-4ea9-9ebc-26590fc8ba5d None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:29.039749 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:29.070349 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb2c84-f292-4c51-aec5-f037218b94da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:29.071348 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb2c84-f292-4c51-aec5-f037218b94da None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:29.071747 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb2c84-f292-4c51-aec5-f037218b94da None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:29.072127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb2c84-f292-4c51-aec5-f037218b94da None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:29.072912 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bb2c84-f292-4c51-aec5-f037218b94da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:29.073224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bb2c84-f292-4c51-aec5-f037218b94da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:29.075508 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bb2c84-f292-4c51-aec5-f037218b94da None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:29.095445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 511/1035] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:29.102132 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04e165-9ec8-4c95-97d2-548560ab31bd None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:29.102946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04e165-9ec8-4c95-97d2-548560ab31bd None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:29.102946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04e165-9ec8-4c95-97d2-548560ab31bd None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:29.104308 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 525/1036] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:59:29.104925 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb2c84-f292-4c51-aec5-f037218b94da None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:29.110589 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3cd2e-91af-488e-a528-607363694bf6 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:29.111191 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3cd2e-91af-488e-a528-607363694bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:29.111757 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3cd2e-91af-488e-a528-607363694bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:29.142787 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-19f3cd2e-91af-488e-a528-607363694bf6 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:29.145669 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-19f3cd2e-91af-488e-a528-607363694bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:29.263749 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 526/1037] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:29.265507 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb2c84-f292-4c51-aec5-f037218b94da None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:29.290914 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3cd2e-91af-488e-a528-607363694bf6 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:29.326098 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ad75b-bb19-4cbc-9a5a-d73b2c4fca4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:29.326744 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ad75b-bb19-4cbc-9a5a-d73b2c4fca4b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:29.327027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ad75b-bb19-4cbc-9a5a-d73b2c4fca4b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:29.327306 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ad75b-bb19-4cbc-9a5a-d73b2c4fca4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:29.338962 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ad75b-bb19-4cbc-9a5a-d73b2c4fca4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:29.339672 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ad75b-bb19-4cbc-9a5a-d73b2c4fca4b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:29.365914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-103ad75b-bb19-4cbc-9a5a-d73b2c4fca4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:59:29.365914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:29.365914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:29.365914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:29.365914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:29.365914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:29.365914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:29.365914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:29.365914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:29.365914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:29.365914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:29.365914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:29.365914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:59:29.365914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:59:29.365914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:59:29.367299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:59:29.367299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:29.367299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:29.367299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:29.367299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:29.367299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:29.367299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:29.367299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:29.367299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:29.367299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:29.367299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:29.367299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:29.367299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:29.367299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:29.367299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:29.367299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:29.367299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:29.369126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:59:29.369126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:59:29.369126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:59:29.369126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:59:29.369126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:59:29.369126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:59:29.369126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:59:29.369126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:29.373012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 512/1038] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:59:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:29.374404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ad75b-bb19-4cbc-9a5a-d73b2c4fca4b None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:29.457834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:29.473938 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3cd2e-91af-488e-a528-607363694bf6 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:29.503777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:29.531335 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9b3e7-7128-4333-adfc-55671379d1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:29.531981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9b3e7-7128-4333-adfc-55671379d1ea None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:29.532269 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9b3e7-7128-4333-adfc-55671379d1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:29.533099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9b3e7-7128-4333-adfc-55671379d1ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:29.547936 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9b3e7-7128-4333-adfc-55671379d1ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:29.553739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-18f9b3e7-7128-4333-adfc-55671379d1ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:59:29.553739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:29.553739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:29.553739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:29.553739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:29.553739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:29.553739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:29.553739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:29.553739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:29.553739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:29.553739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:29.553739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:29.553739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:29.553739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:29.553739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:29.553739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:29.555570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:29.555570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:29.555570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:29.555570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:29.555570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:29.555570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:29.555570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:29.555570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:29.555570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:29.555570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:29.555570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:29.555570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:29.555570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:29.555570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:29.555570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:29.555570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:29.557614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:29.557614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:29.557614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:29.557614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:29.557614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:29.557614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:29.557614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:29.557614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:29.557614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:29.557614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:29.557614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:59:29.557614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:29.560564 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 527/1039] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:59:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:29.561733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9b3e7-7128-4333-adfc-55671379d1ea None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:29.579168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ad75b-bb19-4cbc-9a5a-d73b2c4fca4b None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:29.647866 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f6a6d-11ae-4649-a86d-292dfa767dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:29.648880 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f6a6d-11ae-4649-a86d-292dfa767dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:29.649337 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f6a6d-11ae-4649-a86d-292dfa767dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:29.649811 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f6a6d-11ae-4649-a86d-292dfa767dd8 None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:29.650902 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f6a6d-11ae-4649-a86d-292dfa767dd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:29.651400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f6a6d-11ae-4649-a86d-292dfa767dd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:29.653901 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f6a6d-11ae-4649-a86d-292dfa767dd8 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:29.674743 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 513/1040] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:59:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:29.676884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8837bb8-7666-408f-9920-68e04165126e None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:29.677343 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8837bb8-7666-408f-9920-68e04165126e None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:29.677790 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8837bb8-7666-408f-9920-68e04165126e None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:29.679835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 514/1041] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:29.685302 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8837bb8-7666-408f-9920-68e04165126e None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:29.718986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9b3e7-7128-4333-adfc-55671379d1ea None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:29.772300 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f20f97-52dc-4ac8-8c8a-f795d3d154fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:29.773086 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f20f97-52dc-4ac8-8c8a-f795d3d154fe None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:29.773373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f20f97-52dc-4ac8-8c8a-f795d3d154fe None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:29.773757 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f20f97-52dc-4ac8-8c8a-f795d3d154fe None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:29.775429 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f20f97-52dc-4ac8-8c8a-f795d3d154fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:29.775429 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f20f97-52dc-4ac8-8c8a-f795d3d154fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:29.777144 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f20f97-52dc-4ac8-8c8a-f795d3d154fe None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:29.791508 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 528/1042] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:59:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:29.793378 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f5bf6-4d1f-42f7-bfe4-fdf93d600aa4 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:29.793739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f5bf6-4d1f-42f7-bfe4-fdf93d600aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:29.794047 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f5bf6-4d1f-42f7-bfe4-fdf93d600aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:29.816379 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-ea2f5bf6-4d1f-42f7-bfe4-fdf93d600aa4 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:29.818135 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ea2f5bf6-4d1f-42f7-bfe4-fdf93d600aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:29.837679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:29.844665 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8837bb8-7666-408f-9920-68e04165126e None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:29.871119 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:29.904378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b81a73-deaf-425a-9df1-e7eae108bc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:29.905429 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b81a73-deaf-425a-9df1-e7eae108bc85 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:29.905884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b81a73-deaf-425a-9df1-e7eae108bc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:29.906340 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b81a73-deaf-425a-9df1-e7eae108bc85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:29.907835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b81a73-deaf-425a-9df1-e7eae108bc85 None admin] RBAC: Adding query filter params (role.id=66135f6413e94a77b7c0762b52a4d38e, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=3d9be74621d14c94b95ae94947e5abbd) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:29.908384 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b81a73-deaf-425a-9df1-e7eae108bc85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:29.911133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b81a73-deaf-425a-9df1-e7eae108bc85 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:29.920622 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 515/1043] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:59:29 2022] GET /identity/v3/role_assignments?user.id=3d9be74621d14c94b95ae94947e5abbd&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=66135f6413e94a77b7c0762b52a4d38e => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:29.922362 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b81a73-deaf-425a-9df1-e7eae108bc85 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:29.945321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 529/1044] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:29.976039 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f5bf6-4d1f-42f7-bfe4-fdf93d600aa4 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:30.135141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b81a73-deaf-425a-9df1-e7eae108bc85 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:30.149551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f5bf6-4d1f-42f7-bfe4-fdf93d600aa4 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:30.201918 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47051432-05e0-4eeb-848e-88aaf45e4839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:30.202711 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47051432-05e0-4eeb-848e-88aaf45e4839 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:30.202985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47051432-05e0-4eeb-848e-88aaf45e4839 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:30.203385 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47051432-05e0-4eeb-848e-88aaf45e4839 None admin] PATH_INFO: `/v3/users/34c379b6c1d249e0bd0162443e045900` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:30.222909 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47051432-05e0-4eeb-848e-88aaf45e4839 None admin] RBAC: Authorizing `identity:get_user(user_id=34c379b6c1d249e0bd0162443e045900)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:30.224589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47051432-05e0-4eeb-848e-88aaf45e4839 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:30.226781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 516/1045] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:59:29 2022] GET /identity/v3/users/34c379b6c1d249e0bd0162443e045900 => generated 285 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:30.228751 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f54c6-1eeb-46d0-b080-6f2f10ded20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:30.229728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f54c6-1eeb-46d0-b080-6f2f10ded20d None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:30.230176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f54c6-1eeb-46d0-b080-6f2f10ded20d None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:30.230612 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f54c6-1eeb-46d0-b080-6f2f10ded20d None admin] PATH_INFO: `/v3/services/image` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:30.231677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f54c6-1eeb-46d0-b080-6f2f10ded20d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:30.233057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47051432-05e0-4eeb-848e-88aaf45e4839 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:30.234117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f54c6-1eeb-46d0-b080-6f2f10ded20d None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:30.251782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-d95f54c6-1eeb-46d0-b080-6f2f10ded20d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 14:59:30.251782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:30.251782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:59:30.251782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:59:30.251782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:59:30.251782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:59:30.251782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:59:30.251782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:30.251782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 14:59:30.251782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:30.251782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:59:30.251782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:30.251782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:30.251782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:30.251782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:30.251782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:30.251782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:30.254435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:30.254435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:30.254435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:30.254435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:30.254435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:30.254435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:30.254435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:59:30.254435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:59:30.254435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:59:30.254435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:59:30.254435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:30.254435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:30.254435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:30.254435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:30.254435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:30.254435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:30.256048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:30.256048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:30.256048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:30.256048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:30.256048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:30.256048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:30.256048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:30.256048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:30.256048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:30.256048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:30.256048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:59:30.256048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:30.256048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 14:59:30.256048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:30.256048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 530/1046] 10.222.0.93 () {56 vars in 1105 bytes} [Mon Aug 1 14:59:29 2022] GET /identity/v3/services/image => generated 86 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:30.262411 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f54c6-1eeb-46d0-b080-6f2f10ded20d None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:30.283024 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:30.315977 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:30.402561 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47051432-05e0-4eeb-848e-88aaf45e4839 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:30.438891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f54c6-1eeb-46d0-b080-6f2f10ded20d None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:30.456671 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057596b6-d58d-4edb-9f43-9582d384b761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:30.457494 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057596b6-d58d-4edb-9f43-9582d384b761 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:30.457832 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057596b6-d58d-4edb-9f43-9582d384b761 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:30.458169 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057596b6-d58d-4edb-9f43-9582d384b761 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:30.501159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d96bf5-e158-43a0-a6df-6a72c1c0c0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:30.502449 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d96bf5-e158-43a0-a6df-6a72c1c0c0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:30.502987 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d96bf5-e158-43a0-a6df-6a72c1c0c0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:30.503403 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d96bf5-e158-43a0-a6df-6a72c1c0c0e2 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:30.504592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d96bf5-e158-43a0-a6df-6a72c1c0c0e2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:30.505008 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d96bf5-e158-43a0-a6df-6a72c1c0c0e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:30.506945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d96bf5-e158-43a0-a6df-6a72c1c0c0e2 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:30.513787 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 531/1047] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:30.520979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d96bf5-e158-43a0-a6df-6a72c1c0c0e2 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:30.556118 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057596b6-d58d-4edb-9f43-9582d384b761 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:30.557753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057596b6-d58d-4edb-9f43-9582d384b761 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:30.584657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:30.654050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:30.664028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d96bf5-e158-43a0-a6df-6a72c1c0c0e2 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:30.704395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 517/1048] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:59:30 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:59:30.705740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057596b6-d58d-4edb-9f43-9582d384b761 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:30.717824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50706378-0be6-4cb9-b0ad-d42c5aa0d9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:30.718705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50706378-0be6-4cb9-b0ad-d42c5aa0d9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:30.719162 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50706378-0be6-4cb9-b0ad-d42c5aa0d9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:30.719696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50706378-0be6-4cb9-b0ad-d42c5aa0d9f4 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:30.720755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50706378-0be6-4cb9-b0ad-d42c5aa0d9f4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:30.721229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50706378-0be6-4cb9-b0ad-d42c5aa0d9f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:30.723801 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50706378-0be6-4cb9-b0ad-d42c5aa0d9f4 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:30.731113 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 532/1049] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:30.737515 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50706378-0be6-4cb9-b0ad-d42c5aa0d9f4 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:30.848525 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057596b6-d58d-4edb-9f43-9582d384b761 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:30.878237 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:30.891513 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50706378-0be6-4cb9-b0ad-d42c5aa0d9f4 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:30.896495 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcbb47-549e-42a5-851e-e53dd7680532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:30.897246 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcbb47-549e-42a5-851e-e53dd7680532 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:30.897523 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcbb47-549e-42a5-851e-e53dd7680532 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:30.897871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcbb47-549e-42a5-851e-e53dd7680532 None admin] PATH_INFO: `/v3/projects/76d25a71651f42008bf47d6986eff811` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:30.899211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcbb47-549e-42a5-851e-e53dd7680532 None admin] RBAC: Authorizing `identity:get_project(project_id=76d25a71651f42008bf47d6986eff811)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:30.900694 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcbb47-549e-42a5-851e-e53dd7680532 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:30.903171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 518/1050] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:59:30 2022] GET /identity/v3/projects/76d25a71651f42008bf47d6986eff811 => generated 311 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:30.910327 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcbb47-549e-42a5-851e-e53dd7680532 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:30.959413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3103817-d75d-4f46-b35f-4d5d2d3c4257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:30.960425 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3103817-d75d-4f46-b35f-4d5d2d3c4257 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:30.960937 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3103817-d75d-4f46-b35f-4d5d2d3c4257 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:30.961394 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3103817-d75d-4f46-b35f-4d5d2d3c4257 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:30.962744 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3103817-d75d-4f46-b35f-4d5d2d3c4257 None admin] RBAC: Adding query filter params (interface=public, service_id=8e3094c20aaf4133aee26d8b9c203973, region_id=RegionOne) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:30.963304 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3103817-d75d-4f46-b35f-4d5d2d3c4257 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:30.965966 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3103817-d75d-4f46-b35f-4d5d2d3c4257 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:30.975886 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 533/1051] 10.222.0.93 () {56 vars in 1251 bytes} [Mon Aug 1 14:59:30 2022] GET /identity/v3/endpoints?service_id=8e3094c20aaf4133aee26d8b9c203973&interface=public®ion_id=RegionOne => generated 194 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:30.982638 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3103817-d75d-4f46-b35f-4d5d2d3c4257 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:31.011955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:31.051889 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcbb47-549e-42a5-851e-e53dd7680532 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:31.092098 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608fcbc-5c09-4d91-818f-0f32a89ebaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:31.092754 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608fcbc-5c09-4d91-818f-0f32a89ebaf8 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:31.093035 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608fcbc-5c09-4d91-818f-0f32a89ebaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:31.093306 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608fcbc-5c09-4d91-818f-0f32a89ebaf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:31.094108 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2608fcbc-5c09-4d91-818f-0f32a89ebaf8 None admin] RBAC: Adding query filter params (role.id=5456887a306649bf80a9ea7bd4282e97, scope.project.id=76d25a71651f42008bf47d6986eff811, user.id=34c379b6c1d249e0bd0162443e045900) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:31.094420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2608fcbc-5c09-4d91-818f-0f32a89ebaf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:31.095916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2608fcbc-5c09-4d91-818f-0f32a89ebaf8 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:31.102535 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 519/1052] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:59:30 2022] GET /identity/v3/role_assignments?user.id=34c379b6c1d249e0bd0162443e045900&scope.project.id=76d25a71651f42008bf47d6986eff811&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:31.103984 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3103817-d75d-4f46-b35f-4d5d2d3c4257 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:31.151289 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebe408-d518-4ab3-ada5-71968a5cabf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:31.152305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebe408-d518-4ab3-ada5-71968a5cabf5 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:31.152825 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebe408-d518-4ab3-ada5-71968a5cabf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:31.153248 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebe408-d518-4ab3-ada5-71968a5cabf5 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:31.154119 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebe408-d518-4ab3-ada5-71968a5cabf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:31.156127 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebe408-d518-4ab3-ada5-71968a5cabf5 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:31.164826 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 534/1053] 10.222.0.93 () {56 vars in 1087 bytes} [Mon Aug 1 14:59:30 2022] GET /identity/v3/services => generated 1957 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:59:32.473087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f2f4f-5082-4d69-ac8b-321579b7a3a7 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:32.473917 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f2f4f-5082-4d69-ac8b-321579b7a3a7 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:32.473917 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f2f4f-5082-4d69-ac8b-321579b7a3a7 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:32.475865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 520/1054] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:59:32.482770 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d23fb-3cb6-44cd-b5d5-378a004730a6 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:32.483303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d23fb-3cb6-44cd-b5d5-378a004730a6 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:32.483844 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d23fb-3cb6-44cd-b5d5-378a004730a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:32.515017 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-1f8d23fb-3cb6-44cd-b5d5-378a004730a6 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:32.517085 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-1f8d23fb-3cb6-44cd-b5d5-378a004730a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:32.616076 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 535/1055] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:32.641960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f2f4f-5082-4d69-ac8b-321579b7a3a7 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:32.722864 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:32.834130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f2f4f-5082-4d69-ac8b-321579b7a3a7 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:32.900426 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f303a92-6219-4396-a51b-97e9c15150a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:32.901545 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f303a92-6219-4396-a51b-97e9c15150a5 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:32.901999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f303a92-6219-4396-a51b-97e9c15150a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:32.902452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f303a92-6219-4396-a51b-97e9c15150a5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:32.922474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f303a92-6219-4396-a51b-97e9c15150a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:32.932019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-7f303a92-6219-4396-a51b-97e9c15150a5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:59:32.932019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:32.932019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:32.932019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:32.932019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:32.932019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:32.932019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:32.932019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:32.932019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:32.932019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:32.932019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:32.932019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:32.932019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:32.932019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:32.932019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:32.932019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:32.941210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:32.941210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:32.941210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:32.941210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:32.941210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:32.941210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:32.941210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:32.941210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:32.941210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:32.941210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:32.941210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:32.941210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:32.941210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:32.941210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:32.941210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:32.941210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:32.943249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:32.943249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:32.943249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:32.943249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:32.943249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:32.943249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:32.943249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:32.943249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:32.943249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:32.943249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:32.943249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:59:32.943249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:32.943249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 521/1056] 10.222.0.93 () {56 vars in 1120 bytes} [Mon Aug 1 14:59:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:32.949073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d23fb-3cb6-44cd-b5d5-378a004730a6 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:32.994826 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:33.164222 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d23fb-3cb6-44cd-b5d5-378a004730a6 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:33.232756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacf0a7-5461-4a71-8f6c-70bede97cff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:33.234049 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacf0a7-5461-4a71-8f6c-70bede97cff4 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:33.234617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacf0a7-5461-4a71-8f6c-70bede97cff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:33.235912 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacf0a7-5461-4a71-8f6c-70bede97cff4 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:33.237837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacf0a7-5461-4a71-8f6c-70bede97cff4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:33.238471 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacf0a7-5461-4a71-8f6c-70bede97cff4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:33.241423 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacf0a7-5461-4a71-8f6c-70bede97cff4 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:33.271731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 536/1057] 10.222.0.93 () {56 vars in 1115 bytes} [Mon Aug 1 14:59:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:33.274480 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f303a92-6219-4396-a51b-97e9c15150a5 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:33.500286 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f303a92-6219-4396-a51b-97e9c15150a5 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:33.525899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:33.577632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a833ef1-3962-4d21-ac3b-60eca1ff5483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:33.578880 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a833ef1-3962-4d21-ac3b-60eca1ff5483 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:33.579518 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a833ef1-3962-4d21-ac3b-60eca1ff5483 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:33.580155 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a833ef1-3962-4d21-ac3b-60eca1ff5483 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:33.606682 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a833ef1-3962-4d21-ac3b-60eca1ff5483 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:33.610034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a833ef1-3962-4d21-ac3b-60eca1ff5483 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:33.647736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-9a833ef1-3962-4d21-ac3b-60eca1ff5483 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:33.647736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:33.647736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:33.647736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:33.647736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:33.647736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:33.647736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:33.647736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:33.647736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:33.647736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:33.647736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:33.647736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:33.647736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:59:33.647736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:59:33.647736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:59:33.648881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:59:33.648881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:33.648881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:33.648881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:33.648881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:33.648881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:33.648881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:33.648881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:33.648881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:33.648881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:33.648881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:33.648881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:33.648881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:33.648881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:33.648881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:33.648881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:33.648881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:33.649838 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:59:33.649838 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:59:33.649838 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:33.649838 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:33.650798 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 522/1058] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:33.657848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacf0a7-5461-4a71-8f6c-70bede97cff4 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:33.819799 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:33.866690 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacf0a7-5461-4a71-8f6c-70bede97cff4 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:33.902550 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9cae5-8690-4b7f-b451-123973ab0809 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:33.902987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9cae5-8690-4b7f-b451-123973ab0809 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:33.903410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9cae5-8690-4b7f-b451-123973ab0809 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:33.906128 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 523/1059] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:59:33.914950 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257848ce-3d9f-4ad4-8188-4ba8f3772c0c None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:33.915387 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257848ce-3d9f-4ad4-8188-4ba8f3772c0c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:33.915846 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257848ce-3d9f-4ad4-8188-4ba8f3772c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:33.942037 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbb8ef-5f37-4061-b8b4-9a399db36847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:33.943030 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbb8ef-5f37-4061-b8b4-9a399db36847 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:33.943516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbb8ef-5f37-4061-b8b4-9a399db36847 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:33.943985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbb8ef-5f37-4061-b8b4-9a399db36847 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:33.945292 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cbb8ef-5f37-4061-b8b4-9a399db36847 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:33.945780 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cbb8ef-5f37-4061-b8b4-9a399db36847 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:33.945944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-257848ce-3d9f-4ad4-8188-4ba8f3772c0c None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:33.948714 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-257848ce-3d9f-4ad4-8188-4ba8f3772c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:33.949182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cbb8ef-5f37-4061-b8b4-9a399db36847 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:33.982172 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 537/1060] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:33.984688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbb8ef-5f37-4061-b8b4-9a399db36847 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:34.094559 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 524/1061] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:59:34.121171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9ea2b-3412-439a-a617-8dce3cea1b07 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:34.121647 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9ea2b-3412-439a-a617-8dce3cea1b07 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:34.122132 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9ea2b-3412-439a-a617-8dce3cea1b07 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:34.125092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 525/1062] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:59:34.134755 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28e9cc-a648-4a18-8f1b-e12451b9d8df None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:34.135237 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28e9cc-a648-4a18-8f1b-e12451b9d8df None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:34.135715 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28e9cc-a648-4a18-8f1b-e12451b9d8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:34.168546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-8b28e9cc-a648-4a18-8f1b-e12451b9d8df None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:34.170611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-8b28e9cc-a648-4a18-8f1b-e12451b9d8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:34.202976 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbb8ef-5f37-4061-b8b4-9a399db36847 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:34.272929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdff7e-b23f-4f14-822c-822801b516ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:34.274931 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdff7e-b23f-4f14-822c-822801b516ff None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:34.275412 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdff7e-b23f-4f14-822c-822801b516ff None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:34.275888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdff7e-b23f-4f14-822c-822801b516ff None admin] PATH_INFO: `/v3/users/nova` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:34.291632 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bdff7e-b23f-4f14-822c-822801b516ff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:34.294442 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bdff7e-b23f-4f14-822c-822801b516ff None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:34.299305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:34.306803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 526/1063] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:59:34.307931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28e9cc-a648-4a18-8f1b-e12451b9d8df None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:34.322446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-49bdff7e-b23f-4f14-822c-822801b516ff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:59:34.322446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:34.322446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:34.322446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:34.322446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:34.322446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:34.322446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:34.322446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:34.322446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:34.322446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:34.322446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:34.322446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:34.322446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:59:34.322446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:59:34.322446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:59:34.322446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:59:34.324168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:34.324168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:34.324168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:59:34.324168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:59:34.324168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:59:34.324168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:59:34.324168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:34.324168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:34.324168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:34.324168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:34.324168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:34.324168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:34.324168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:34.324168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:34.324168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:34.324168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:34.324168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:34.324168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:34.332251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:34.332251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:34.332251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:59:34.332251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:59:34.332251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:59:34.332251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:59:34.332251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:59:34.332251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:59:34.332251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:59:34.332251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:34.332251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 538/1064] 10.222.0.93 () {56 vars in 1093 bytes} [Mon Aug 1 14:59:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:34.333618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdff7e-b23f-4f14-822c-822801b516ff None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:34.369011 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:34.532062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28e9cc-a648-4a18-8f1b-e12451b9d8df None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:34.555680 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdff7e-b23f-4f14-822c-822801b516ff None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:34.613676 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c6a50-4ac8-4f24-a7ee-f21fd32fce37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:34.614462 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c6a50-4ac8-4f24-a7ee-f21fd32fce37 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:34.614766 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c6a50-4ac8-4f24-a7ee-f21fd32fce37 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:34.615080 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c6a50-4ac8-4f24-a7ee-f21fd32fce37 None admin] PATH_INFO: `/v3/services/image` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:34.615713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c6a50-4ac8-4f24-a7ee-f21fd32fce37 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:34.618339 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c6a50-4ac8-4f24-a7ee-f21fd32fce37 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:34.627304 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d8681-08eb-4cf4-9e35-9ba8f25b7301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:34.627896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d8681-08eb-4cf4-9e35-9ba8f25b7301 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:34.628710 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d8681-08eb-4cf4-9e35-9ba8f25b7301 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:34.628985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d8681-08eb-4cf4-9e35-9ba8f25b7301 None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:34.629800 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d8681-08eb-4cf4-9e35-9ba8f25b7301 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:34.630192 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d8681-08eb-4cf4-9e35-9ba8f25b7301 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:34.632621 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d8681-08eb-4cf4-9e35-9ba8f25b7301 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:34.633418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-2e6c6a50-4ac8-4f24-a7ee-f21fd32fce37 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 14:59:34.633418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:34.633418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 14:59:34.633418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 14:59:34.633418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 14:59:34.633418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 14:59:34.633418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 14:59:34.633418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:34.633418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 14:59:34.633418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:34.633418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 14:59:34.633418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:34.633418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:34.633418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:34.633418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:34.633418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:34.633418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:34.635860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:34.635860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:34.635860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:34.635860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:34.635860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:34.635860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:34.635860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 14:59:34.635860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 14:59:34.635860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 14:59:34.635860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 14:59:34.635860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:34.635860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:34.635860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:34.635860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:34.635860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:34.635860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:34.638553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:34.638553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:34.638553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:34.638553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:34.638553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:34.638553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:34.638553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:34.638553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:34.638553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:34.638553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:34.638553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 14:59:34.638553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 14:59:34.638553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 14:59:34.638553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:34.638553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 527/1065] 10.222.0.93 () {56 vars in 1105 bytes} [Mon Aug 1 14:59:34 2022] GET /identity/v3/services/image => generated 86 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:34.640283 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c6a50-4ac8-4f24-a7ee-f21fd32fce37 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:34.666941 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 539/1066] 10.222.0.93 () {56 vars in 1133 bytes} [Mon Aug 1 14:59:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:34.668126 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d8681-08eb-4cf4-9e35-9ba8f25b7301 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:34.779051 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:34.806409 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:34.825464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c6a50-4ac8-4f24-a7ee-f21fd32fce37 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:34.871463 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d8681-08eb-4cf4-9e35-9ba8f25b7301 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:34.903294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f8248-552b-48cf-856d-aaa5de0983f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:34.905116 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f8248-552b-48cf-856d-aaa5de0983f3 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:34.905595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f8248-552b-48cf-856d-aaa5de0983f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:34.906132 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f8248-552b-48cf-856d-aaa5de0983f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:34.927190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f8248-552b-48cf-856d-aaa5de0983f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:34.932050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4cd22-7981-429a-b67d-411b17318c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:34.932655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4cd22-7981-429a-b67d-411b17318c08 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:34.933115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4cd22-7981-429a-b67d-411b17318c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:34.933561 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4cd22-7981-429a-b67d-411b17318c08 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:34.935666 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4cd22-7981-429a-b67d-411b17318c08 None admin] RBAC: Adding query filter params (name=image) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:34.937258 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4cd22-7981-429a-b67d-411b17318c08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:34.937438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-e93f8248-552b-48cf-856d-aaa5de0983f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:59:34.937438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:34.937438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:34.937438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:34.937438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:34.937438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:34.937438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:34.937438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:34.937438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:34.937438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:34.937438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:34.937438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:34.937438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:34.937438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:34.937438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:34.937438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:34.939189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:34.939189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:34.939189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:34.939189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:34.939189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:34.939189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:34.939189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:34.939189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:34.939189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:34.939189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:34.939189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:34.939189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:34.939189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:34.939189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:34.939189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:34.939189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:34.941245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:34.941245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:34.941245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:34.941245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:34.941245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:34.941245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:34.941245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:34.941245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:34.941245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:34.941245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:34.941245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:59:34.941245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:34.941245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 528/1067] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:59:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:34.941245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f8248-552b-48cf-856d-aaa5de0983f3 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:34.942646 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4cd22-7981-429a-b67d-411b17318c08 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:34.956294 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 540/1068] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:34.956294 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4cd22-7981-429a-b67d-411b17318c08 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:35.167829 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4cd22-7981-429a-b67d-411b17318c08 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:35.171723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f8248-552b-48cf-856d-aaa5de0983f3 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:35.208144 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:35.229329 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aac8a9-9f93-4fb5-8870-3cac65d2b008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:35.229995 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aac8a9-9f93-4fb5-8870-3cac65d2b008 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:35.230275 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aac8a9-9f93-4fb5-8870-3cac65d2b008 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:35.230547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aac8a9-9f93-4fb5-8870-3cac65d2b008 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:35.232993 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142c4a3-2cd1-44f2-b800-469f8b464f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:35.234697 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142c4a3-2cd1-44f2-b800-469f8b464f9f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:35.234697 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142c4a3-2cd1-44f2-b800-469f8b464f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:35.235043 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142c4a3-2cd1-44f2-b800-469f8b464f9f None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:35.235995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142c4a3-2cd1-44f2-b800-469f8b464f9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:35.236417 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142c4a3-2cd1-44f2-b800-469f8b464f9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:35.238509 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142c4a3-2cd1-44f2-b800-469f8b464f9f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:35.251584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aac8a9-9f93-4fb5-8870-3cac65d2b008 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:35.254476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aac8a9-9f93-4fb5-8870-3cac65d2b008 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:35.263951 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 541/1069] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:59:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:35.264788 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142c4a3-2cd1-44f2-b800-469f8b464f9f None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:35.288947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-47aac8a9-9f93-4fb5-8870-3cac65d2b008 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:35.288947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:35.288947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:35.288947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:35.288947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:35.288947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:35.288947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:35.288947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:35.288947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:35.288947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:35.288947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:35.288947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:35.288947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:59:35.288947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:59:35.288947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:59:35.290980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:59:35.290980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:35.290980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:35.290980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:35.290980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:35.290980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:35.290980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:35.290980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:35.290980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:35.290980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:35.290980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:35.290980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:35.290980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:35.290980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:35.290980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:35.290980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:35.290980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:35.292924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:59:35.292924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:59:35.292924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:35.292924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:35.292924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 529/1070] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:35.292924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aac8a9-9f93-4fb5-8870-3cac65d2b008 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:35.297768 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:35.464485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142c4a3-2cd1-44f2-b800-469f8b464f9f None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:35.506756 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aac8a9-9f93-4fb5-8870-3cac65d2b008 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:35.542596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930a97c-8bd8-4c07-bc31-13bf512cc783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:35.543501 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930a97c-8bd8-4c07-bc31-13bf512cc783 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:35.543792 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930a97c-8bd8-4c07-bc31-13bf512cc783 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:35.545541 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930a97c-8bd8-4c07-bc31-13bf512cc783 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:35.546555 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930a97c-8bd8-4c07-bc31-13bf512cc783 None admin] RBAC: Adding query filter params (type=image) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:35.547763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930a97c-8bd8-4c07-bc31-13bf512cc783 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:35.551545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930a97c-8bd8-4c07-bc31-13bf512cc783 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:35.561819 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 542/1071] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:35.563280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930a97c-8bd8-4c07-bc31-13bf512cc783 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:35.590509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8491140-a370-449f-a539-c2926efffe79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:35.591877 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8491140-a370-449f-a539-c2926efffe79 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:35.593227 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8491140-a370-449f-a539-c2926efffe79 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:35.594520 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8491140-a370-449f-a539-c2926efffe79 None admin] PATH_INFO: `/v3/users/34c379b6c1d249e0bd0162443e045900` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:35.597949 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8491140-a370-449f-a539-c2926efffe79 None admin] RBAC: Authorizing `identity:get_user(user_id=34c379b6c1d249e0bd0162443e045900)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:35.602231 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8491140-a370-449f-a539-c2926efffe79 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:35.605002 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 530/1072] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:59:35 2022] GET /identity/v3/users/34c379b6c1d249e0bd0162443e045900 => generated 285 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:35.605892 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8491140-a370-449f-a539-c2926efffe79 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:35.662500 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:35.738797 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:35.790319 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930a97c-8bd8-4c07-bc31-13bf512cc783 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:35.800760 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8491140-a370-449f-a539-c2926efffe79 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:35.857222 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3c47f-3e73-4a4e-bab0-497685d3fd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:35.858288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3c47f-3e73-4a4e-bab0-497685d3fd6a None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:35.860685 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3c47f-3e73-4a4e-bab0-497685d3fd6a None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:35.860685 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3c47f-3e73-4a4e-bab0-497685d3fd6a None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:35.861321 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b42f1-383f-4a50-8f95-1e237fa4c6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:35.861508 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3c47f-3e73-4a4e-bab0-497685d3fd6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:35.861942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3c47f-3e73-4a4e-bab0-497685d3fd6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:35.862739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b42f1-383f-4a50-8f95-1e237fa4c6d3 None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:35.863113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b42f1-383f-4a50-8f95-1e237fa4c6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:35.863476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b42f1-383f-4a50-8f95-1e237fa4c6d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:35.864310 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b42f1-383f-4a50-8f95-1e237fa4c6d3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:35.865083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3c47f-3e73-4a4e-bab0-497685d3fd6a None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:35.866814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b42f1-383f-4a50-8f95-1e237fa4c6d3 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:35.901219 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 543/1073] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:35.901219 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3c47f-3e73-4a4e-bab0-497685d3fd6a None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:35.912090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 531/1074] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 14:59:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:59:35.912090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b42f1-383f-4a50-8f95-1e237fa4c6d3 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:36.104983 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3c47f-3e73-4a4e-bab0-497685d3fd6a None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:36.130902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:36.189203 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b42f1-383f-4a50-8f95-1e237fa4c6d3 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:36.191329 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741af3f-34c1-4c78-a3b7-d585a78d88a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:36.195240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741af3f-34c1-4c78-a3b7-d585a78d88a6 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:36.195731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741af3f-34c1-4c78-a3b7-d585a78d88a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:36.196222 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741af3f-34c1-4c78-a3b7-d585a78d88a6 None admin] PATH_INFO: `/v3/projects/76d25a71651f42008bf47d6986eff811` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:36.199397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741af3f-34c1-4c78-a3b7-d585a78d88a6 None admin] RBAC: Authorizing `identity:get_project(project_id=76d25a71651f42008bf47d6986eff811)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:36.202446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741af3f-34c1-4c78-a3b7-d585a78d88a6 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:36.205670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 544/1075] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:59:35 2022] GET /identity/v3/projects/76d25a71651f42008bf47d6986eff811 => generated 311 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:36.220246 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741af3f-34c1-4c78-a3b7-d585a78d88a6 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:36.248877 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:36.273318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdde60e-4335-4cbc-acef-12d62ccdc037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:36.274427 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdde60e-4335-4cbc-acef-12d62ccdc037 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:36.275007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdde60e-4335-4cbc-acef-12d62ccdc037 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:36.275787 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdde60e-4335-4cbc-acef-12d62ccdc037 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:36.288960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdde60e-4335-4cbc-acef-12d62ccdc037 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:36.290952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdde60e-4335-4cbc-acef-12d62ccdc037 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:36.312141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-6cdde60e-4335-4cbc-acef-12d62ccdc037 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:59:36.312141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:36.312141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:36.312141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:36.312141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:36.312141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:36.312141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:36.312141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:36.312141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:36.312141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:36.312141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:36.312141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:36.312141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:59:36.312141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:59:36.312141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:59:36.314091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:59:36.314091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:36.314091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:36.314091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:36.314091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:36.314091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:36.314091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:36.314091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:36.314091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:36.314091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:36.314091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:36.314091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:36.314091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:36.314091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:36.314091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:36.314091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:36.314091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:36.316433 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:59:36.316433 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:59:36.316433 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:59:36.316433 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:59:36.316433 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:59:36.316433 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:59:36.316433 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:59:36.316433 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:36.320979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 532/1076] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:59:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:36.323455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdde60e-4335-4cbc-acef-12d62ccdc037 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:36.458819 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741af3f-34c1-4c78-a3b7-d585a78d88a6 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:36.538353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcc179-c6ab-4ae8-b64b-5525c70a1ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:36.539252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcc179-c6ab-4ae8-b64b-5525c70a1ea6 None admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:36.539743 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcc179-c6ab-4ae8-b64b-5525c70a1ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:36.540230 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcc179-c6ab-4ae8-b64b-5525c70a1ea6 None admin] PATH_INFO: `/v3/projects/76d25a71651f42008bf47d6986eff811/users/34c379b6c1d249e0bd0162443e045900/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:36.544667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dcc179-c6ab-4ae8-b64b-5525c70a1ea6 None admin] RBAC: Authorizing `identity:create_grant(project_id=76d25a71651f42008bf47d6986eff811, user_id=34c379b6c1d249e0bd0162443e045900, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:36.547252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dcc179-c6ab-4ae8-b64b-5525c70a1ea6 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:36.568205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdde60e-4335-4cbc-acef-12d62ccdc037 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:36.576605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 545/1077] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:59:36 2022] PUT /identity/v3/projects/76d25a71651f42008bf47d6986eff811/users/34c379b6c1d249e0bd0162443e045900/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:59:36.648251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64996e4c-ec48-46e9-b20c-861874a825ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:36.649776 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64996e4c-ec48-46e9-b20c-861874a825ed None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:36.649776 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64996e4c-ec48-46e9-b20c-861874a825ed None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:36.649776 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64996e4c-ec48-46e9-b20c-861874a825ed None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:36.654761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64996e4c-ec48-46e9-b20c-861874a825ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:36.654761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64996e4c-ec48-46e9-b20c-861874a825ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:36.654761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64996e4c-ec48-46e9-b20c-861874a825ed None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:36.676961 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 533/1078] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:59:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:36.691342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcc179-c6ab-4ae8-b64b-5525c70a1ea6 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:36.765802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:36.890862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcc179-c6ab-4ae8-b64b-5525c70a1ea6 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:36.962733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29c2d2-76e7-4090-9427-d4ff4103aa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:36.963920 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29c2d2-76e7-4090-9427-d4ff4103aa46 None admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:36.964584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29c2d2-76e7-4090-9427-d4ff4103aa46 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:36.964930 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29c2d2-76e7-4090-9427-d4ff4103aa46 None admin] PATH_INFO: `/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/3d9be74621d14c94b95ae94947e5abbd/roles/66135f6413e94a77b7c0762b52a4d38e` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:36.968867 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29c2d2-76e7-4090-9427-d4ff4103aa46 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac9f7c74597642b0acecc956429ac4d8, user_id=3d9be74621d14c94b95ae94947e5abbd, role_id=66135f6413e94a77b7c0762b52a4d38e)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:36.971626 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29c2d2-76e7-4090-9427-d4ff4103aa46 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:36.996604 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 546/1079] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:59:36 2022] PUT /identity/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/3d9be74621d14c94b95ae94947e5abbd/roles/66135f6413e94a77b7c0762b52a4d38e => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:59:38.846859 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c128d-8af7-44ef-87fa-05fa16a14539 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:38.847734 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c128d-8af7-44ef-87fa-05fa16a14539 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:38.847833 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c128d-8af7-44ef-87fa-05fa16a14539 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:38.852622 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 534/1080] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:38.858377 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e5dcf-f6c2-4fd6-baad-dbe087ec9917 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:38.859193 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e5dcf-f6c2-4fd6-baad-dbe087ec9917 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:38.859353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e5dcf-f6c2-4fd6-baad-dbe087ec9917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:38.892765 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-391e5dcf-f6c2-4fd6-baad-dbe087ec9917 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:38.895303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-391e5dcf-f6c2-4fd6-baad-dbe087ec9917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:39.032544 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 547/1081] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:59:39.076658 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c128d-8af7-44ef-87fa-05fa16a14539 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:39.139883 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:39.225620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c128d-8af7-44ef-87fa-05fa16a14539 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:39.281869 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ade8e-9728-4410-9f70-75e6d9070b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:39.282702 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ade8e-9728-4410-9f70-75e6d9070b77 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:39.283117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ade8e-9728-4410-9f70-75e6d9070b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:39.283531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ade8e-9728-4410-9f70-75e6d9070b77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:39.395863 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ade8e-9728-4410-9f70-75e6d9070b77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:39.397834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ade8e-9728-4410-9f70-75e6d9070b77 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:39.471886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:39.580537 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 535/1082] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:59:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 14:59:39.584772 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e5dcf-f6c2-4fd6-baad-dbe087ec9917 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:39.618715 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c565a82-34ff-4ede-90d6-d39721cf8d55 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:39.619046 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c565a82-34ff-4ede-90d6-d39721cf8d55 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:39.619283 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c565a82-34ff-4ede-90d6-d39721cf8d55 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:39.620004 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:39.622402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 536/1083] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:39.628731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893d377-de9e-478c-90d1-73cd7249299e None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:39.629031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893d377-de9e-478c-90d1-73cd7249299e None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:39.629318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893d377-de9e-478c-90d1-73cd7249299e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:39.653887 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-8893d377-de9e-478c-90d1-73cd7249299e None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:39.655327 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-8893d377-de9e-478c-90d1-73cd7249299e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:39.727297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e5dcf-f6c2-4fd6-baad-dbe087ec9917 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:39.746682 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 537/1084] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:59:39.771188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893d377-de9e-478c-90d1-73cd7249299e None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:39.791017 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27b8d4-09dd-4754-a898-94c2a585a8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:39.791756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27b8d4-09dd-4754-a898-94c2a585a8ec None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:39.792044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27b8d4-09dd-4754-a898-94c2a585a8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:39.792326 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27b8d4-09dd-4754-a898-94c2a585a8ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:39.807699 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27b8d4-09dd-4754-a898-94c2a585a8ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:39.810187 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27b8d4-09dd-4754-a898-94c2a585a8ec None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:39.830480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-cd27b8d4-09dd-4754-a898-94c2a585a8ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:39.830480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:39.830480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:39.830480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:39.830480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:39.830480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:39.830480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:39.830480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:39.830480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:39.830480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:39.830480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:39.830480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:39.830480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:59:39.830480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:59:39.830480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:59:39.833150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:59:39.833150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:39.833150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:39.833150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:39.833150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:39.833150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:39.833150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:39.833150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:39.833150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:39.833150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:39.833150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:39.833150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:39.833150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:39.833150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:39.833150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:39.833150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:39.833150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:39.835795 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:59:39.835795 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:59:39.835795 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:39.835795 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:39.840005 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 548/1085] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:39.841310 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27b8d4-09dd-4754-a898-94c2a585a8ec None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:39.870277 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:39.922500 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893d377-de9e-478c-90d1-73cd7249299e None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:39.991448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:39.992220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56ab30-8ffb-43b4-ae87-38af2190cdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:39.993153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56ab30-8ffb-43b4-ae87-38af2190cdb6 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:39.993443 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56ab30-8ffb-43b4-ae87-38af2190cdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:39.993939 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56ab30-8ffb-43b4-ae87-38af2190cdb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:40.025151 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56ab30-8ffb-43b4-ae87-38af2190cdb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:40.035051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-1c56ab30-8ffb-43b4-ae87-38af2190cdb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:59:40.035051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:40.035051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:40.035051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:40.035051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:40.035051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:40.035051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:40.035051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:40.035051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:40.035051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:40.035051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:40.035051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:40.035051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:40.035051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:40.035051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:40.035051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:40.036372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:40.036372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:40.036372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:40.036372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:40.036372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:40.036372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:40.036372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:40.036372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:40.036372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:40.036372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:40.036372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:40.036372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:40.036372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:40.036372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:40.036372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:40.036372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:40.037769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:40.037769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:40.037769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:40.037769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:40.037769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:40.037769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:40.037769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:40.037769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:40.037769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:40.037769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:40.037769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:59:40.037769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:40.037769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 538/1086] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 14:59:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:40.048106 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56ab30-8ffb-43b4-ae87-38af2190cdb6 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:40.053354 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27b8d4-09dd-4754-a898-94c2a585a8ec None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:40.107668 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe08301-d91f-4bf2-8429-67368e87e55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:40.108843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe08301-d91f-4bf2-8429-67368e87e55f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:40.109288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe08301-d91f-4bf2-8429-67368e87e55f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:40.109917 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe08301-d91f-4bf2-8429-67368e87e55f None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:40.111325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe08301-d91f-4bf2-8429-67368e87e55f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:40.111685 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe08301-d91f-4bf2-8429-67368e87e55f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:40.114659 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe08301-d91f-4bf2-8429-67368e87e55f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:40.143022 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 549/1087] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:40.143022 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f822c-d8a9-4c40-a725-aa04b093ad3d None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:40.143022 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f822c-d8a9-4c40-a725-aa04b093ad3d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:40.143632 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f822c-d8a9-4c40-a725-aa04b093ad3d None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:40.146483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 550/1088] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:40.155960 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec90d2-3ce1-4d53-8621-67ebd2c7fd24 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:40.156507 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec90d2-3ce1-4d53-8621-67ebd2c7fd24 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:40.156974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec90d2-3ce1-4d53-8621-67ebd2c7fd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:40.190021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-85ec90d2-3ce1-4d53-8621-67ebd2c7fd24 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:40.192621 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-85ec90d2-3ce1-4d53-8621-67ebd2c7fd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:40.245295 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56ab30-8ffb-43b4-ae87-38af2190cdb6 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:40.289537 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:40.298621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd7131-5e2d-4823-92f8-b35018389bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:40.299530 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd7131-5e2d-4823-92f8-b35018389bce None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:40.299784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd7131-5e2d-4823-92f8-b35018389bce None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:40.300071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd7131-5e2d-4823-92f8-b35018389bce None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:40.301360 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd7131-5e2d-4823-92f8-b35018389bce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:40.301360 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd7131-5e2d-4823-92f8-b35018389bce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:40.302684 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd7131-5e2d-4823-92f8-b35018389bce None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:40.313089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 539/1089] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 14:59:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:40.322465 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd7131-5e2d-4823-92f8-b35018389bce None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:40.338363 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 551/1090] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:40.362404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec90d2-3ce1-4d53-8621-67ebd2c7fd24 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:40.469394 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:40.496742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd7131-5e2d-4823-92f8-b35018389bce None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:40.532704 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec90d2-3ce1-4d53-8621-67ebd2c7fd24 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:40.546983 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316a439-13cc-43b8-892a-833e707d69cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:40.548383 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316a439-13cc-43b8-892a-833e707d69cf None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:40.548818 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316a439-13cc-43b8-892a-833e707d69cf None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:40.549264 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316a439-13cc-43b8-892a-833e707d69cf None admin] PATH_INFO: `/v3/users/34c379b6c1d249e0bd0162443e045900` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:40.551873 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316a439-13cc-43b8-892a-833e707d69cf None admin] RBAC: Authorizing `identity:get_user(user_id=34c379b6c1d249e0bd0162443e045900)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:40.554689 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316a439-13cc-43b8-892a-833e707d69cf None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:40.557885 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 540/1091] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:59:40 2022] GET /identity/v3/users/34c379b6c1d249e0bd0162443e045900 => generated 285 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:40.567550 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316a439-13cc-43b8-892a-833e707d69cf None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:40.579622 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7a0d0-7489-4a9e-8c29-120ca9fa8a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:40.580925 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7a0d0-7489-4a9e-8c29-120ca9fa8a16 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:40.581200 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7a0d0-7489-4a9e-8c29-120ca9fa8a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:40.581554 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7a0d0-7489-4a9e-8c29-120ca9fa8a16 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:40.595086 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7a0d0-7489-4a9e-8c29-120ca9fa8a16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:40.600667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-afa7a0d0-7489-4a9e-8c29-120ca9fa8a16 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:59:40.600667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:40.600667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:40.600667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:40.600667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:40.600667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:40.600667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:40.600667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:40.600667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:40.600667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:40.600667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:40.600667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:40.600667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:40.600667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:40.600667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:40.600667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:40.602571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:40.602571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:40.602571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:40.602571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:40.602571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:40.602571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:40.602571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:40.602571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:40.602571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:40.602571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:40.602571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:40.602571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:40.602571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:40.602571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:40.602571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:40.602571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:40.604288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:40.604288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:40.604288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:40.604288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:40.604288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:40.604288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:40.604288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:40.604288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:40.604288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:40.604288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:40.604288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 14:59:40.604288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:40.604288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 552/1092] 10.222.0.93 () {56 vars in 1120 bytes} [Mon Aug 1 14:59:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:40.611430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7a0d0-7489-4a9e-8c29-120ca9fa8a16 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:40.648600 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:40.696207 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316a439-13cc-43b8-892a-833e707d69cf None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:40.736586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a552f-07ed-457c-bd08-780629fc3fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:40.737617 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a552f-07ed-457c-bd08-780629fc3fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:40.737890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a552f-07ed-457c-bd08-780629fc3fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:40.738161 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a552f-07ed-457c-bd08-780629fc3fb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:40.775468 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7a0d0-7489-4a9e-8c29-120ca9fa8a16 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:40.811121 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:40.821806 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad654b-64a2-436f-83d5-47bae1900612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:40.822593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad654b-64a2-436f-83d5-47bae1900612 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:40.822956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad654b-64a2-436f-83d5-47bae1900612 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:40.823366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad654b-64a2-436f-83d5-47bae1900612 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:40.824217 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad654b-64a2-436f-83d5-47bae1900612 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:40.824575 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad654b-64a2-436f-83d5-47bae1900612 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:40.825969 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad654b-64a2-436f-83d5-47bae1900612 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:40.836097 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 553/1093] 10.222.0.93 () {56 vars in 1115 bytes} [Mon Aug 1 14:59:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:40.843385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad654b-64a2-436f-83d5-47bae1900612 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:40.847564 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a552f-07ed-457c-bd08-780629fc3fb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:40.850387 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a552f-07ed-457c-bd08-780629fc3fb0 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:40.926561 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:40.976551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad654b-64a2-436f-83d5-47bae1900612 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:40.983528 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 541/1094] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:59:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 14:59:40.990531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a552f-07ed-457c-bd08-780629fc3fb0 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:41.014446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ba030-c46f-4156-b97d-7fa7c5a24314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:41.015096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ba030-c46f-4156-b97d-7fa7c5a24314 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:41.015368 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ba030-c46f-4156-b97d-7fa7c5a24314 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:41.015736 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ba030-c46f-4156-b97d-7fa7c5a24314 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:41.027847 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ba030-c46f-4156-b97d-7fa7c5a24314 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:41.029395 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ba030-c46f-4156-b97d-7fa7c5a24314 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:41.047670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-f71ba030-c46f-4156-b97d-7fa7c5a24314 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:41.047670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:41.047670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:41.047670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:41.047670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:41.047670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:41.047670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:41.047670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:41.047670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:41.047670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:41.047670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:41.047670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:41.047670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:59:41.047670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:59:41.047670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:59:41.048830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:59:41.048830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:41.048830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:41.048830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:41.048830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:41.048830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:41.048830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:41.048830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:41.048830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:41.048830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:41.048830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:41.048830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:41.048830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:41.048830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:41.048830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:41.048830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:41.048830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:41.050831 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:59:41.050831 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:59:41.050831 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:41.050831 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:41.050831 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 554/1095] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:41.055424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ba030-c46f-4156-b97d-7fa7c5a24314 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:41.106379 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:41.112750 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a552f-07ed-457c-bd08-780629fc3fb0 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:41.179761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df399084-e08a-4e15-b926-4116195cc630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:41.180613 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df399084-e08a-4e15-b926-4116195cc630 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:41.181021 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df399084-e08a-4e15-b926-4116195cc630 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:41.181470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df399084-e08a-4e15-b926-4116195cc630 None admin] PATH_INFO: `/v3/projects/76d25a71651f42008bf47d6986eff811` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:41.183475 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df399084-e08a-4e15-b926-4116195cc630 None admin] RBAC: Authorizing `identity:get_project(project_id=76d25a71651f42008bf47d6986eff811)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:41.186024 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df399084-e08a-4e15-b926-4116195cc630 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:41.193964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 542/1096] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:59:40 2022] GET /identity/v3/projects/76d25a71651f42008bf47d6986eff811 => generated 311 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:41.195357 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df399084-e08a-4e15-b926-4116195cc630 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:41.197584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ba030-c46f-4156-b97d-7fa7c5a24314 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:41.243601 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c23b8d-782f-4293-b4e2-6f489cd0e90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:41.244709 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c23b8d-782f-4293-b4e2-6f489cd0e90f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:41.245222 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c23b8d-782f-4293-b4e2-6f489cd0e90f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:41.245721 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c23b8d-782f-4293-b4e2-6f489cd0e90f None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:41.246640 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c23b8d-782f-4293-b4e2-6f489cd0e90f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:41.247205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c23b8d-782f-4293-b4e2-6f489cd0e90f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:41.249786 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c23b8d-782f-4293-b4e2-6f489cd0e90f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:41.253114 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:41.269593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 555/1097] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:41.275714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c23b8d-782f-4293-b4e2-6f489cd0e90f None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:41.373527 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df399084-e08a-4e15-b926-4116195cc630 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:41.432882 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d9615-6104-4ead-bfcb-339b22df7eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:41.433669 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d9615-6104-4ead-bfcb-339b22df7eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:41.434039 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d9615-6104-4ead-bfcb-339b22df7eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:41.434487 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d9615-6104-4ead-bfcb-339b22df7eb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:41.435442 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d9615-6104-4ead-bfcb-339b22df7eb2 None admin] RBAC: Adding query filter params (role.id=5456887a306649bf80a9ea7bd4282e97, scope.project.id=76d25a71651f42008bf47d6986eff811, user.id=34c379b6c1d249e0bd0162443e045900) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:41.435795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d9615-6104-4ead-bfcb-339b22df7eb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:41.437022 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c23b8d-782f-4293-b4e2-6f489cd0e90f None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:41.437652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d9615-6104-4ead-bfcb-339b22df7eb2 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:41.440782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:41.446737 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 543/1098] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:59:41 2022] GET /identity/v3/role_assignments?user.id=34c379b6c1d249e0bd0162443e045900&scope.project.id=76d25a71651f42008bf47d6986eff811&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:41.512438 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fa7f3-2efa-4f99-8004-fbf38aa83f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:41.513554 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fa7f3-2efa-4f99-8004-fbf38aa83f82 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:41.514058 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fa7f3-2efa-4f99-8004-fbf38aa83f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:41.514517 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fa7f3-2efa-4f99-8004-fbf38aa83f82 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:41.529194 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fa7f3-2efa-4f99-8004-fbf38aa83f82 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:41.531532 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fa7f3-2efa-4f99-8004-fbf38aa83f82 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:41.553655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-435fa7f3-2efa-4f99-8004-fbf38aa83f82 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:59:41.553655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:41.553655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:41.553655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:41.553655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:41.553655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:41.553655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:41.553655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:41.553655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:41.553655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:41.553655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:41.553655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:41.553655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 14:59:41.553655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 14:59:41.553655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 14:59:41.553655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 14:59:41.555610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:41.555610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:41.555610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 14:59:41.555610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:59:41.555610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 14:59:41.555610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 14:59:41.555610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:41.555610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:41.555610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:41.555610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:41.555610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:41.555610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:41.555610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:41.555610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:41.555610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:41.555610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:41.555610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:41.555610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:41.557619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:41.557619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:41.557619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 14:59:41.557619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 14:59:41.557619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 14:59:41.557619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 14:59:41.557619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 14:59:41.557619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 14:59:41.557619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 14:59:41.557619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 14:59:41.557619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 556/1099] 10.222.0.93 () {56 vars in 1093 bytes} [Mon Aug 1 14:59:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:41.562661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d9615-6104-4ead-bfcb-339b22df7eb2 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:41.741199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:41.752005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d9615-6104-4ead-bfcb-339b22df7eb2 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:41.806418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c40d1d-f7ee-4f36-a0fb-d1c5a1e67a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:41.806967 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c40d1d-f7ee-4f36-a0fb-d1c5a1e67a31 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:41.807241 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c40d1d-f7ee-4f36-a0fb-d1c5a1e67a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:41.807509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c40d1d-f7ee-4f36-a0fb-d1c5a1e67a31 None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:41.808320 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c40d1d-f7ee-4f36-a0fb-d1c5a1e67a31 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:41.808680 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c40d1d-f7ee-4f36-a0fb-d1c5a1e67a31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:41.810797 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c40d1d-f7ee-4f36-a0fb-d1c5a1e67a31 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:41.833455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 544/1100] 10.222.0.93 () {56 vars in 1133 bytes} [Mon Aug 1 14:59:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:41.839903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fa7f3-2efa-4f99-8004-fbf38aa83f82 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:42.027378 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fa7f3-2efa-4f99-8004-fbf38aa83f82 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:42.070980 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b845a-e48d-4200-acfb-b6535db5fb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:42.071530 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b845a-e48d-4200-acfb-b6535db5fb45 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:42.071807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b845a-e48d-4200-acfb-b6535db5fb45 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:42.072073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b845a-e48d-4200-acfb-b6535db5fb45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:42.109193 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:42.154855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b845a-e48d-4200-acfb-b6535db5fb45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:42.156339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b845a-e48d-4200-acfb-b6535db5fb45 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:42.276838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 557/1101] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:59:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:59:42.283059 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c40d1d-f7ee-4f36-a0fb-d1c5a1e67a31 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:42.408211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c40d1d-f7ee-4f36-a0fb-d1c5a1e67a31 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:42.448171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4f5e3-400f-4936-a1e1-be0a44a1dd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:42.448990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4f5e3-400f-4936-a1e1-be0a44a1dd40 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:42.449270 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4f5e3-400f-4936-a1e1-be0a44a1dd40 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:42.449538 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4f5e3-400f-4936-a1e1-be0a44a1dd40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:42.461608 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4f5e3-400f-4936-a1e1-be0a44a1dd40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:42.463219 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4f5e3-400f-4936-a1e1-be0a44a1dd40 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:42.481369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-b2d4f5e3-400f-4936-a1e1-be0a44a1dd40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:42.481369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:42.481369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:42.481369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:42.481369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:42.481369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:42.481369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:42.481369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:42.481369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:42.481369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:42.481369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:42.481369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:42.481369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:59:42.481369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:59:42.481369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:59:42.482543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:59:42.482543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:42.482543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:42.482543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:42.482543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:42.482543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:42.482543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:42.482543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:42.482543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:42.482543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:42.482543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:42.482543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:42.482543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:42.482543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:42.482543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:42.482543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:42.482543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:42.484453 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:59:42.484453 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:59:42.484453 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:42.484453 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:42.484453 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 545/1102] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:42.491063 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b845a-e48d-4200-acfb-b6535db5fb45 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:42.568477 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:42.624706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b845a-e48d-4200-acfb-b6535db5fb45 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:42.676819 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139aa9b0-dfe1-4b80-be8a-c413ec0b0cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:42.677462 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139aa9b0-dfe1-4b80-be8a-c413ec0b0cde None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:42.677747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139aa9b0-dfe1-4b80-be8a-c413ec0b0cde None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:42.678097 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139aa9b0-dfe1-4b80-be8a-c413ec0b0cde None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:42.679218 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139aa9b0-dfe1-4b80-be8a-c413ec0b0cde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:42.679552 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139aa9b0-dfe1-4b80-be8a-c413ec0b0cde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:42.681673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139aa9b0-dfe1-4b80-be8a-c413ec0b0cde None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:42.701509 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 558/1103] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 14:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:42.709401 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4f5e3-400f-4936-a1e1-be0a44a1dd40 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:42.713745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:42.844244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4f5e3-400f-4936-a1e1-be0a44a1dd40 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:42.887584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c037e-087b-4d6b-89b7-29bc19e6f418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:42.888235 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c037e-087b-4d6b-89b7-29bc19e6f418 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:42.888567 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c037e-087b-4d6b-89b7-29bc19e6f418 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:42.888921 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c037e-087b-4d6b-89b7-29bc19e6f418 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:42.899176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c037e-087b-4d6b-89b7-29bc19e6f418 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:42.901287 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c037e-087b-4d6b-89b7-29bc19e6f418 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:42.914410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-918c037e-087b-4d6b-89b7-29bc19e6f418 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:59:42.914410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:42.914410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:42.914410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:42.914410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:42.914410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:42.914410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:42.914410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:42.914410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:42.914410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:42.914410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:42.914410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:42.914410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 14:59:42.914410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 14:59:42.914410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 14:59:42.917752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 14:59:42.917752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:42.917752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:42.917752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:42.917752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:42.917752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:42.917752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:42.917752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:42.917752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:42.917752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:42.917752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:42.917752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:42.917752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:42.917752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:42.917752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:42.917752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:42.917752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:42.920156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 14:59:42.920156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 14:59:42.920156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 14:59:42.920156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 14:59:42.920156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 14:59:42.920156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 14:59:42.920156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 14:59:42.920156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 14:59:42.921883 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 546/1104] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 14:59:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:42.923903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139aa9b0-dfe1-4b80-be8a-c413ec0b0cde None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:43.073902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139aa9b0-dfe1-4b80-be8a-c413ec0b0cde None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:43.113976 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:43.135824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03af1e-1806-4f3b-bebc-eebf7d624406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:43.136789 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03af1e-1806-4f3b-bebc-eebf7d624406 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:43.137267 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03af1e-1806-4f3b-bebc-eebf7d624406 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:43.137738 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03af1e-1806-4f3b-bebc-eebf7d624406 None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:43.139193 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03af1e-1806-4f3b-bebc-eebf7d624406 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:43.139713 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03af1e-1806-4f3b-bebc-eebf7d624406 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:43.142559 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03af1e-1806-4f3b-bebc-eebf7d624406 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:43.167692 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 559/1105] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 14:59:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:43.178537 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c037e-087b-4d6b-89b7-29bc19e6f418 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:43.262791 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:43.325154 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c037e-087b-4d6b-89b7-29bc19e6f418 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:43.374844 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8101b5-8a65-4220-b185-131f282d2041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:43.375465 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8101b5-8a65-4220-b185-131f282d2041 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:43.375743 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8101b5-8a65-4220-b185-131f282d2041 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:43.376020 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8101b5-8a65-4220-b185-131f282d2041 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:43.376920 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8101b5-8a65-4220-b185-131f282d2041 None admin] RBAC: Adding query filter params (role.id=66135f6413e94a77b7c0762b52a4d38e, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=3d9be74621d14c94b95ae94947e5abbd) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:43.377226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8101b5-8a65-4220-b185-131f282d2041 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:43.379117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8101b5-8a65-4220-b185-131f282d2041 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:43.386806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 547/1106] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:59:43 2022] GET /identity/v3/role_assignments?user.id=3d9be74621d14c94b95ae94947e5abbd&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=66135f6413e94a77b7c0762b52a4d38e => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:43.841399 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed480628-49d4-4fba-bf9f-ded0e47e38ff None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:43.841858 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed480628-49d4-4fba-bf9f-ded0e47e38ff None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:43.842274 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed480628-49d4-4fba-bf9f-ded0e47e38ff None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:43.844627 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 560/1107] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:43.852086 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd4b02-261c-4434-858e-5e7ee4605beb None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:43.852615 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd4b02-261c-4434-858e-5e7ee4605beb None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:43.853081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd4b02-261c-4434-858e-5e7ee4605beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:43.875033 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-23bd4b02-261c-4434-858e-5e7ee4605beb None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:43.876278 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-23bd4b02-261c-4434-858e-5e7ee4605beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:43.954575 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 548/1108] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:43.980046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed480628-49d4-4fba-bf9f-ded0e47e38ff None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:44.137589 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed480628-49d4-4fba-bf9f-ded0e47e38ff None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:44.202372 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bc22d-55e5-4d98-8f2c-7a66e7e91876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:44.203557 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bc22d-55e5-4d98-8f2c-7a66e7e91876 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:44.204280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bc22d-55e5-4d98-8f2c-7a66e7e91876 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:44.204878 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bc22d-55e5-4d98-8f2c-7a66e7e91876 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:44.207371 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bc22d-55e5-4d98-8f2c-7a66e7e91876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:44.210482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bc22d-55e5-4d98-8f2c-7a66e7e91876 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:44.213800 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 561/1109] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:44.222849 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd4b02-261c-4434-858e-5e7ee4605beb None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:44.316413 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:44.389019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd4b02-261c-4434-858e-5e7ee4605beb None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:44.455232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076f7a7-3793-43d8-931f-14b821ec8dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:44.455938 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076f7a7-3793-43d8-931f-14b821ec8dc1 None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:44.456294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076f7a7-3793-43d8-931f-14b821ec8dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:44.456647 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076f7a7-3793-43d8-931f-14b821ec8dc1 None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:44.457366 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076f7a7-3793-43d8-931f-14b821ec8dc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:44.459097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076f7a7-3793-43d8-931f-14b821ec8dc1 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:44.493476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 549/1110] 10.222.0.93 () {60 vars in 1132 bytes} [Mon Aug 1 14:59:44 2022] POST /identity/v3/users => generated 286 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:59:46.768581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e732380-2923-4317-8741-4902f5751044 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:46.769234 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e732380-2923-4317-8741-4902f5751044 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:46.769483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e732380-2923-4317-8741-4902f5751044 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:46.772154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 562/1111] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:46.780766 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6ac52-acbd-449c-b153-8021ddce6011 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:46.781612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6ac52-acbd-449c-b153-8021ddce6011 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:46.781780 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6ac52-acbd-449c-b153-8021ddce6011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:46.804313 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-00d6ac52-acbd-449c-b153-8021ddce6011 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:46.805778 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-00d6ac52-acbd-449c-b153-8021ddce6011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:46.893256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 550/1112] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:59:46.928878 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e732380-2923-4317-8741-4902f5751044 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:47.013765 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:47.094915 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e732380-2923-4317-8741-4902f5751044 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:47.158036 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c1e5d-c5f1-4683-9b72-c5ceabb29cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:47.159502 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c1e5d-c5f1-4683-9b72-c5ceabb29cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:47.159952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c1e5d-c5f1-4683-9b72-c5ceabb29cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:47.160334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c1e5d-c5f1-4683-9b72-c5ceabb29cc6 None admin] PATH_INFO: `/v3/roles/e880d7a2b8c74bb1beb5015723a4e3d7` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:47.162255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c1e5d-c5f1-4683-9b72-c5ceabb29cc6 None admin] RBAC: Authorizing `identity:get_role(role_id=e880d7a2b8c74bb1beb5015723a4e3d7)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:47.163899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c1e5d-c5f1-4683-9b72-c5ceabb29cc6 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:47.165700 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 563/1113] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:59:46 2022] GET /identity/v3/roles/e880d7a2b8c74bb1beb5015723a4e3d7 => generated 223 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:47.173477 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6ac52-acbd-449c-b153-8021ddce6011 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:47.269927 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:47.381549 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6ac52-acbd-449c-b153-8021ddce6011 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:47.445409 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe499dc-758e-4c08-bb25-03e635826438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:47.446527 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe499dc-758e-4c08-bb25-03e635826438 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:47.447019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe499dc-758e-4c08-bb25-03e635826438 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:47.447483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe499dc-758e-4c08-bb25-03e635826438 None admin] PATH_INFO: `/v3/users/bb579239a5924c6394aad7f43ef472b9` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:47.467792 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe499dc-758e-4c08-bb25-03e635826438 None admin] RBAC: Authorizing `identity:get_user(user_id=bb579239a5924c6394aad7f43ef472b9)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:47.469763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe499dc-758e-4c08-bb25-03e635826438 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:47.471664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 551/1114] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:59:47 2022] GET /identity/v3/users/bb579239a5924c6394aad7f43ef472b9 => generated 286 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:47.479578 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c1e5d-c5f1-4683-9b72-c5ceabb29cc6 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:47.681129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c1e5d-c5f1-4683-9b72-c5ceabb29cc6 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:47.705780 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:47.730836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355724c-6bac-429e-8e3d-f72ec026d03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:47.731655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355724c-6bac-429e-8e3d-f72ec026d03c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:47.732040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355724c-6bac-429e-8e3d-f72ec026d03c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:47.732339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355724c-6bac-429e-8e3d-f72ec026d03c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:47.820550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355724c-6bac-429e-8e3d-f72ec026d03c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:47.822294 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355724c-6bac-429e-8e3d-f72ec026d03c None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:47.993673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 564/1115] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:59:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:59:48.001464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe499dc-758e-4c08-bb25-03e635826438 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:48.189058 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe499dc-758e-4c08-bb25-03e635826438 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:48.192305 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:48.241440 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf89f4-6f83-47ee-88db-800aaff9da40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:48.242092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf89f4-6f83-47ee-88db-800aaff9da40 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:48.242371 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf89f4-6f83-47ee-88db-800aaff9da40 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:48.242646 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf89f4-6f83-47ee-88db-800aaff9da40 None admin] PATH_INFO: `/v3/projects/76d25a71651f42008bf47d6986eff811` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:48.244005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf89f4-6f83-47ee-88db-800aaff9da40 None admin] RBAC: Authorizing `identity:get_project(project_id=76d25a71651f42008bf47d6986eff811)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:48.245694 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf89f4-6f83-47ee-88db-800aaff9da40 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:48.247448 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 552/1116] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:59:48 2022] GET /identity/v3/projects/76d25a71651f42008bf47d6986eff811 => generated 311 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:48.255422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355724c-6bac-429e-8e3d-f72ec026d03c None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:48.289551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:48.403415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355724c-6bac-429e-8e3d-f72ec026d03c None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:48.447173 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4c8a0-1c66-4a8b-94f2-20faba064e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:48.447871 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4c8a0-1c66-4a8b-94f2-20faba064e35 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:48.448197 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4c8a0-1c66-4a8b-94f2-20faba064e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:48.448554 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4c8a0-1c66-4a8b-94f2-20faba064e35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:48.449405 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4c8a0-1c66-4a8b-94f2-20faba064e35 None admin] RBAC: Adding query filter params (role.id=e880d7a2b8c74bb1beb5015723a4e3d7, scope.project.id=76d25a71651f42008bf47d6986eff811, user.id=bb579239a5924c6394aad7f43ef472b9) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:48.449758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4c8a0-1c66-4a8b-94f2-20faba064e35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:48.451287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4c8a0-1c66-4a8b-94f2-20faba064e35 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:48.458549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 565/1117] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:59:48 2022] GET /identity/v3/role_assignments?user.id=bb579239a5924c6394aad7f43ef472b9&scope.project.id=76d25a71651f42008bf47d6986eff811&role.id=e880d7a2b8c74bb1beb5015723a4e3d7 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:50.558603 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cc64d-f262-4334-8476-4802ce51217d None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:50.559293 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cc64d-f262-4334-8476-4802ce51217d None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:50.559293 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cc64d-f262-4334-8476-4802ce51217d None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:50.560533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 553/1118] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:50.566975 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974155b2-c653-4adc-b9c1-9053b50a985f None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:50.567589 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974155b2-c653-4adc-b9c1-9053b50a985f None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:50.567589 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974155b2-c653-4adc-b9c1-9053b50a985f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:50.586452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-974155b2-c653-4adc-b9c1-9053b50a985f None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:50.587667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-974155b2-c653-4adc-b9c1-9053b50a985f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:50.662455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 566/1119] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 14:59:50.685884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cc64d-f262-4334-8476-4802ce51217d None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:50.846930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cc64d-f262-4334-8476-4802ce51217d None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:50.890669 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dd4e4-1fc8-4f8e-b9fb-d795fcc3c7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:50.891307 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dd4e4-1fc8-4f8e-b9fb-d795fcc3c7dc None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:50.891587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dd4e4-1fc8-4f8e-b9fb-d795fcc3c7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:50.891854 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dd4e4-1fc8-4f8e-b9fb-d795fcc3c7dc None admin] PATH_INFO: `/v3/roles/e880d7a2b8c74bb1beb5015723a4e3d7` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:50.893877 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842dd4e4-1fc8-4f8e-b9fb-d795fcc3c7dc None admin] RBAC: Authorizing `identity:get_role(role_id=e880d7a2b8c74bb1beb5015723a4e3d7)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:50.895262 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842dd4e4-1fc8-4f8e-b9fb-d795fcc3c7dc None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:50.896653 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 554/1120] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:59:50 2022] GET /identity/v3/roles/e880d7a2b8c74bb1beb5015723a4e3d7 => generated 223 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:50.902890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974155b2-c653-4adc-b9c1-9053b50a985f None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:50.918540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:51.009261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974155b2-c653-4adc-b9c1-9053b50a985f None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:51.055788 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8969045-1540-4243-9536-9256c2345428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:51.056543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8969045-1540-4243-9536-9256c2345428 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:51.056877 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8969045-1540-4243-9536-9256c2345428 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:51.057189 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8969045-1540-4243-9536-9256c2345428 None admin] PATH_INFO: `/v3/users/bb579239a5924c6394aad7f43ef472b9` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:51.058659 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8969045-1540-4243-9536-9256c2345428 None admin] RBAC: Authorizing `identity:get_user(user_id=bb579239a5924c6394aad7f43ef472b9)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:51.060412 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8969045-1540-4243-9536-9256c2345428 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:51.062794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 567/1121] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:59:50 2022] GET /identity/v3/users/bb579239a5924c6394aad7f43ef472b9 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:51.072188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dd4e4-1fc8-4f8e-b9fb-d795fcc3c7dc None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:51.119062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:51.231736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dd4e4-1fc8-4f8e-b9fb-d795fcc3c7dc None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:51.280454 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde1954-2d63-40d3-8b4a-a6b864400647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:51.281124 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde1954-2d63-40d3-8b4a-a6b864400647 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:51.281402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde1954-2d63-40d3-8b4a-a6b864400647 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:51.281675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde1954-2d63-40d3-8b4a-a6b864400647 None admin] PATH_INFO: `/v3/projects/76d25a71651f42008bf47d6986eff811` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:51.282859 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde1954-2d63-40d3-8b4a-a6b864400647 None admin] RBAC: Authorizing `identity:get_project(project_id=76d25a71651f42008bf47d6986eff811)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:51.284347 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde1954-2d63-40d3-8b4a-a6b864400647 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:51.286093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 555/1122] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:59:51 2022] GET /identity/v3/projects/76d25a71651f42008bf47d6986eff811 => generated 311 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:51.292369 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8969045-1540-4243-9536-9256c2345428 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:51.388884 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:51.417399 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8969045-1540-4243-9536-9256c2345428 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:51.465748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2186167-ff77-4cbf-97c3-46afa863b58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:51.466684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2186167-ff77-4cbf-97c3-46afa863b58f None admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:51.467005 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2186167-ff77-4cbf-97c3-46afa863b58f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:51.467327 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2186167-ff77-4cbf-97c3-46afa863b58f None admin] PATH_INFO: `/v3/projects/76d25a71651f42008bf47d6986eff811/users/bb579239a5924c6394aad7f43ef472b9/roles/e880d7a2b8c74bb1beb5015723a4e3d7` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:51.470116 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2186167-ff77-4cbf-97c3-46afa863b58f None admin] RBAC: Authorizing `identity:create_grant(project_id=76d25a71651f42008bf47d6986eff811, user_id=bb579239a5924c6394aad7f43ef472b9, role_id=e880d7a2b8c74bb1beb5015723a4e3d7)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:51.472595 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2186167-ff77-4cbf-97c3-46afa863b58f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:51.488570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 568/1123] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 14:59:51 2022] PUT /identity/v3/projects/76d25a71651f42008bf47d6986eff811/users/bb579239a5924c6394aad7f43ef472b9/roles/e880d7a2b8c74bb1beb5015723a4e3d7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:59:53.555333 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069243e-9046-4a7a-ba3b-0d4762a5ef05 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:53.555902 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069243e-9046-4a7a-ba3b-0d4762a5ef05 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:53.555902 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069243e-9046-4a7a-ba3b-0d4762a5ef05 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:53.557821 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 556/1124] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:53.563587 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192fdca-5e5a-4e36-a7eb-c5c5247a4244 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:53.564041 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192fdca-5e5a-4e36-a7eb-c5c5247a4244 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:53.564157 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192fdca-5e5a-4e36-a7eb-c5c5247a4244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:53.582479 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-9192fdca-5e5a-4e36-a7eb-c5c5247a4244 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:53.583668 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-9192fdca-5e5a-4e36-a7eb-c5c5247a4244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:53.652624 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 569/1125] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:53.677284 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069243e-9046-4a7a-ba3b-0d4762a5ef05 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:53.786373 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069243e-9046-4a7a-ba3b-0d4762a5ef05 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:53.794145 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:53.822397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88cd2e-5f39-475a-9327-ca45c35ef1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:53.823048 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88cd2e-5f39-475a-9327-ca45c35ef1b6 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:53.823326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88cd2e-5f39-475a-9327-ca45c35ef1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:53.823599 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88cd2e-5f39-475a-9327-ca45c35ef1b6 None admin] PATH_INFO: `/v3/roles/e880d7a2b8c74bb1beb5015723a4e3d7` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:53.825081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf88cd2e-5f39-475a-9327-ca45c35ef1b6 None admin] RBAC: Authorizing `identity:get_role(role_id=e880d7a2b8c74bb1beb5015723a4e3d7)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:53.826455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf88cd2e-5f39-475a-9327-ca45c35ef1b6 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:53.827804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 557/1126] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:59:53 2022] GET /identity/v3/roles/e880d7a2b8c74bb1beb5015723a4e3d7 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:53.834688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192fdca-5e5a-4e36-a7eb-c5c5247a4244 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:53.908078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:53.945058 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192fdca-5e5a-4e36-a7eb-c5c5247a4244 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:53.998774 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd30634-5c63-4c3b-a0cf-24b086dabdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:53.999857 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd30634-5c63-4c3b-a0cf-24b086dabdf0 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:54.000363 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd30634-5c63-4c3b-a0cf-24b086dabdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:54.000899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd30634-5c63-4c3b-a0cf-24b086dabdf0 None admin] PATH_INFO: `/v3/users/bb579239a5924c6394aad7f43ef472b9` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:54.002970 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd30634-5c63-4c3b-a0cf-24b086dabdf0 None admin] RBAC: Authorizing `identity:get_user(user_id=bb579239a5924c6394aad7f43ef472b9)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:54.005744 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd30634-5c63-4c3b-a0cf-24b086dabdf0 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:54.008819 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 570/1127] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 14:59:53 2022] GET /identity/v3/users/bb579239a5924c6394aad7f43ef472b9 => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:54.014799 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88cd2e-5f39-475a-9327-ca45c35ef1b6 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:54.119784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88cd2e-5f39-475a-9327-ca45c35ef1b6 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:54.153819 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e576d5-6e86-4a27-b714-693c791ad90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:54.154432 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e576d5-6e86-4a27-b714-693c791ad90a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:54.154715 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e576d5-6e86-4a27-b714-693c791ad90a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:54.154989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e576d5-6e86-4a27-b714-693c791ad90a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:54.201607 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:54.229796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e576d5-6e86-4a27-b714-693c791ad90a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:54.231257 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e576d5-6e86-4a27-b714-693c791ad90a None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:54.336651 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 558/1128] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 14:59:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 14:59:54.342906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd30634-5c63-4c3b-a0cf-24b086dabdf0 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:54.462118 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd30634-5c63-4c3b-a0cf-24b086dabdf0 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:54.506903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf24199-2ad5-4fae-aed9-afa8eb49a394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:54.508043 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf24199-2ad5-4fae-aed9-afa8eb49a394 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:54.508318 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf24199-2ad5-4fae-aed9-afa8eb49a394 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:54.508611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf24199-2ad5-4fae-aed9-afa8eb49a394 None admin] PATH_INFO: `/v3/projects/76d25a71651f42008bf47d6986eff811` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:54.510160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf24199-2ad5-4fae-aed9-afa8eb49a394 None admin] RBAC: Authorizing `identity:get_project(project_id=76d25a71651f42008bf47d6986eff811)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:54.511608 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf24199-2ad5-4fae-aed9-afa8eb49a394 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:54.513323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 571/1129] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 14:59:54 2022] GET /identity/v3/projects/76d25a71651f42008bf47d6986eff811 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:54.521360 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e576d5-6e86-4a27-b714-693c791ad90a None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:54.598083 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:54.623691 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e576d5-6e86-4a27-b714-693c791ad90a None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:54.659848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0c77b-3948-44aa-8cef-835385f62827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:54.660457 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0c77b-3948-44aa-8cef-835385f62827 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:54.660762 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0c77b-3948-44aa-8cef-835385f62827 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:54.661027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0c77b-3948-44aa-8cef-835385f62827 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:54.661797 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0c77b-3948-44aa-8cef-835385f62827 None admin] RBAC: Adding query filter params (role.id=e880d7a2b8c74bb1beb5015723a4e3d7, scope.project.id=76d25a71651f42008bf47d6986eff811, user.id=bb579239a5924c6394aad7f43ef472b9) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:54.662098 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0c77b-3948-44aa-8cef-835385f62827 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:54.663519 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0c77b-3948-44aa-8cef-835385f62827 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:54.669805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 559/1130] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 14:59:54 2022] GET /identity/v3/role_assignments?user.id=bb579239a5924c6394aad7f43ef472b9&scope.project.id=76d25a71651f42008bf47d6986eff811&role.id=e880d7a2b8c74bb1beb5015723a4e3d7 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:56.775209 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76506e-d716-4138-a990-ef409f4f185a None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:56.775734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76506e-d716-4138-a990-ef409f4f185a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:56.775734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76506e-d716-4138-a990-ef409f4f185a None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:56.777083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 572/1131] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:59:56.782742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c59f0-7731-4c5d-a62a-b250627b4aaa None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:56.783249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c59f0-7731-4c5d-a62a-b250627b4aaa None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:56.783365 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c59f0-7731-4c5d-a62a-b250627b4aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:56.801851 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-512c59f0-7731-4c5d-a62a-b250627b4aaa None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:56.803075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-512c59f0-7731-4c5d-a62a-b250627b4aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:56.879577 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 560/1132] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:56.903672 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76506e-d716-4138-a990-ef409f4f185a None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:56.913353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:57.012983 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76506e-d716-4138-a990-ef409f4f185a None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:57.047900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97837b19-2a14-462b-91c4-1b04be29246c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:57.048449 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97837b19-2a14-462b-91c4-1b04be29246c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:57.048775 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97837b19-2a14-462b-91c4-1b04be29246c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:57.049610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97837b19-2a14-462b-91c4-1b04be29246c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:57.050785 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97837b19-2a14-462b-91c4-1b04be29246c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:57.052240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97837b19-2a14-462b-91c4-1b04be29246c None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:57.053915 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 573/1133] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:57.060226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c59f0-7731-4c5d-a62a-b250627b4aaa None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:57.130208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:57.164397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c59f0-7731-4c5d-a62a-b250627b4aaa None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:57.200361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cee72-41ef-47f4-b083-ae500f76ad83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:57.201005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cee72-41ef-47f4-b083-ae500f76ad83 None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:57.201285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cee72-41ef-47f4-b083-ae500f76ad83 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:57.201558 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cee72-41ef-47f4-b083-ae500f76ad83 None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:57.202245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00cee72-41ef-47f4-b083-ae500f76ad83 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:57.203675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00cee72-41ef-47f4-b083-ae500f76ad83 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:57.229990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 561/1134] 10.222.0.93 () {60 vars in 1141 bytes} [Mon Aug 1 14:59:57 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:59:59.541600 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf543f-1787-463b-9ad5-8bf856289f4e None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:59.542124 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf543f-1787-463b-9ad5-8bf856289f4e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:59.542124 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf543f-1787-463b-9ad5-8bf856289f4e None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:59.543480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 574/1135] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 14:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:59:59.548889 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fb592-c074-4544-a4d4-35cc2c97b347 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:59.549330 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fb592-c074-4544-a4d4-35cc2c97b347 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:59.549462 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fb592-c074-4544-a4d4-35cc2c97b347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:59.569381 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-619fb592-c074-4544-a4d4-35cc2c97b347 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:59.570590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-619fb592-c074-4544-a4d4-35cc2c97b347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:59.641797 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 562/1136] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 14:59:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 14:59:59.666867 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf543f-1787-463b-9ad5-8bf856289f4e None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:59.748574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:59.781562 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf543f-1787-463b-9ad5-8bf856289f4e None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:59.818496 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bbf75-851b-430b-b813-e746a4fa9c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:59.819128 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bbf75-851b-430b-b813-e746a4fa9c9e None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:59.819402 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bbf75-851b-430b-b813-e746a4fa9c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:59.819771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bbf75-851b-430b-b813-e746a4fa9c9e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:59.821013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bbf75-851b-430b-b813-e746a4fa9c9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:59.822455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bbf75-851b-430b-b813-e746a4fa9c9e None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:59.824071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 575/1137] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 14:59:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:59.830815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619fb592-c074-4544-a4d4-35cc2c97b347 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:59.884738 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:59.940913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619fb592-c074-4544-a4d4-35cc2c97b347 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:59.978121 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfb9b0-9ca9-4655-8a8e-f9e1012ef4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:59.978751 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfb9b0-9ca9-4655-8a8e-f9e1012ef4d0 None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:59.979032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfb9b0-9ca9-4655-8a8e-f9e1012ef4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:59.979308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfb9b0-9ca9-4655-8a8e-f9e1012ef4d0 None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:59.980007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfb9b0-9ca9-4655-8a8e-f9e1012ef4d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:59.981496 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfb9b0-9ca9-4655-8a8e-f9e1012ef4d0 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:00.007525 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 563/1138] 10.222.0.93 () {60 vars in 1132 bytes} [Mon Aug 1 14:59:59 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:00:02.216799 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2eed6-1c4c-44ec-911e-aac6cdd62874 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:02.217584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2eed6-1c4c-44ec-911e-aac6cdd62874 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:02.217584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2eed6-1c4c-44ec-911e-aac6cdd62874 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:02.218921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 576/1139] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:00:02.225252 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff6252-951a-48d4-86b8-2fe5090abe59 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:02.225777 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff6252-951a-48d4-86b8-2fe5090abe59 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:02.225777 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff6252-951a-48d4-86b8-2fe5090abe59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:02.244566 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-36ff6252-951a-48d4-86b8-2fe5090abe59 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:02.245727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-36ff6252-951a-48d4-86b8-2fe5090abe59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:02.327791 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 564/1140] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:00:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:00:02.360836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2eed6-1c4c-44ec-911e-aac6cdd62874 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:02.474926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2eed6-1c4c-44ec-911e-aac6cdd62874 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:02.517137 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d2e6d-7eb3-4180-b7be-ab02439001ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:02.517841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d2e6d-7eb3-4180-b7be-ab02439001ab None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:02.518121 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d2e6d-7eb3-4180-b7be-ab02439001ab None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:02.518391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d2e6d-7eb3-4180-b7be-ab02439001ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:02.531346 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d2e6d-7eb3-4180-b7be-ab02439001ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:02.536950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-c40d2e6d-7eb3-4180-b7be-ab02439001ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:02.536950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:02.536950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:02.536950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:02.536950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:02.536950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:02.536950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:02.536950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:02.536950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:02.536950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:02.536950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:02.536950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:02.536950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:02.536950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:02.536950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:02.536950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:02.539021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:02.539021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:02.539021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:02.539021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:02.539021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:02.539021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:02.539021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:02.539021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:02.539021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:02.539021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:02.539021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:02.539021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:02.539021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:02.539021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:02.539021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:02.539021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:02.541211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:02.541211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:02.541211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:02.541211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:02.541211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:02.541211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:02.541211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:02.541211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:02.541211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:02.541211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:02.541211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:02.541211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:00:02.541211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 577/1141] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 15:00:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:02.546648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff6252-951a-48d4-86b8-2fe5090abe59 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:02.598499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:02.694771 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff6252-951a-48d4-86b8-2fe5090abe59 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:02.737479 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b8dde-51fc-4e13-a0f4-c16eec85eabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:02.738116 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b8dde-51fc-4e13-a0f4-c16eec85eabd None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:02.738395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b8dde-51fc-4e13-a0f4-c16eec85eabd None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:02.738669 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b8dde-51fc-4e13-a0f4-c16eec85eabd None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:02.740911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b8dde-51fc-4e13-a0f4-c16eec85eabd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:02.740911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b8dde-51fc-4e13-a0f4-c16eec85eabd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:02.741253 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b8dde-51fc-4e13-a0f4-c16eec85eabd None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:02.750771 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 565/1142] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 15:00:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:02.757809 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d2e6d-7eb3-4180-b7be-ab02439001ab None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:02.761856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:02.875019 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d2e6d-7eb3-4180-b7be-ab02439001ab None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:02.915982 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f47fc3-22d3-4f4d-9e32-62fffb39ccb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:02.917277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f47fc3-22d3-4f4d-9e32-62fffb39ccb6 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:02.917564 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f47fc3-22d3-4f4d-9e32-62fffb39ccb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:02.917835 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f47fc3-22d3-4f4d-9e32-62fffb39ccb6 None admin] PATH_INFO: `/v3/users/76137b3d6a664fa29caa1c4abfb9d211` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:02.933522 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f47fc3-22d3-4f4d-9e32-62fffb39ccb6 None admin] RBAC: Authorizing `identity:get_user(user_id=76137b3d6a664fa29caa1c4abfb9d211)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:02.934986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f47fc3-22d3-4f4d-9e32-62fffb39ccb6 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:02.936758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 578/1143] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 15:00:02 2022] GET /identity/v3/users/76137b3d6a664fa29caa1c4abfb9d211 => generated 286 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:02.943459 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b8dde-51fc-4e13-a0f4-c16eec85eabd None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:03.061248 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:03.065776 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b8dde-51fc-4e13-a0f4-c16eec85eabd None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:03.110474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24119c9-49fe-4c0d-bccf-b786d9b0a9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:03.111117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24119c9-49fe-4c0d-bccf-b786d9b0a9ae None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:03.111400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24119c9-49fe-4c0d-bccf-b786d9b0a9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:03.111660 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24119c9-49fe-4c0d-bccf-b786d9b0a9ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:03.231358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24119c9-49fe-4c0d-bccf-b786d9b0a9ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:03.234250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24119c9-49fe-4c0d-bccf-b786d9b0a9ae None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:03.356707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:03.361974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 566/1144] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 15:00:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:00:03.371191 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f47fc3-22d3-4f4d-9e32-62fffb39ccb6 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:03.443374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:03.488776 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f47fc3-22d3-4f4d-9e32-62fffb39ccb6 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:03.546211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664ad77-1762-448b-a582-f24e24af093f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:03.547368 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664ad77-1762-448b-a582-f24e24af093f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:03.547863 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664ad77-1762-448b-a582-f24e24af093f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:03.548380 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664ad77-1762-448b-a582-f24e24af093f None admin] PATH_INFO: `/v3/projects/95369f2b03f94749a49041fc7e0a8f6b` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:03.550710 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664ad77-1762-448b-a582-f24e24af093f None admin] RBAC: Authorizing `identity:get_project(project_id=95369f2b03f94749a49041fc7e0a8f6b)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:03.553652 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664ad77-1762-448b-a582-f24e24af093f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:03.557099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 579/1145] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 15:00:03 2022] GET /identity/v3/projects/95369f2b03f94749a49041fc7e0a8f6b => generated 311 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:03.563586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24119c9-49fe-4c0d-bccf-b786d9b0a9ae None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:03.710359 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24119c9-49fe-4c0d-bccf-b786d9b0a9ae None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:03.751297 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122008b3-8aca-4dc1-9a76-c2ab142f3bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:03.752601 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122008b3-8aca-4dc1-9a76-c2ab142f3bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:03.752878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122008b3-8aca-4dc1-9a76-c2ab142f3bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:03.753150 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122008b3-8aca-4dc1-9a76-c2ab142f3bc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:03.753940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122008b3-8aca-4dc1-9a76-c2ab142f3bc1 None admin] RBAC: Adding query filter params (role.id=5456887a306649bf80a9ea7bd4282e97, scope.project.id=95369f2b03f94749a49041fc7e0a8f6b, user.id=76137b3d6a664fa29caa1c4abfb9d211) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:03.754275 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122008b3-8aca-4dc1-9a76-c2ab142f3bc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:03.755726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122008b3-8aca-4dc1-9a76-c2ab142f3bc1 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:03.762106 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 567/1146] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 15:00:03 2022] GET /identity/v3/role_assignments?user.id=76137b3d6a664fa29caa1c4abfb9d211&scope.project.id=95369f2b03f94749a49041fc7e0a8f6b&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:05.981453 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fcd62-08cb-4985-8ca4-a924870eb323 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:05.982025 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fcd62-08cb-4985-8ca4-a924870eb323 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:05.982025 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fcd62-08cb-4985-8ca4-a924870eb323 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:05.983434 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 580/1147] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:00:05.988951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7012a-d833-41e2-8fbf-27cb64e4a64d None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:05.989455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7012a-d833-41e2-8fbf-27cb64e4a64d None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:05.989550 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7012a-d833-41e2-8fbf-27cb64e4a64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:06.009087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-83a7012a-d833-41e2-8fbf-27cb64e4a64d None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:06.010303 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-83a7012a-d833-41e2-8fbf-27cb64e4a64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:06.077452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:06.089248 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 568/1148] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:00:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:00:06.112444 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0fcd62-08cb-4985-8ca4-a924870eb323 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:06.227450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0fcd62-08cb-4985-8ca4-a924870eb323 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:06.255657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:06.264696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a2958-6ff8-4f9c-a09d-09d1aea2a210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:06.265347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a2958-6ff8-4f9c-a09d-09d1aea2a210 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:06.265624 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a2958-6ff8-4f9c-a09d-09d1aea2a210 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:06.265895 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a2958-6ff8-4f9c-a09d-09d1aea2a210 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:06.277797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a2958-6ff8-4f9c-a09d-09d1aea2a210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:06.282758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-e58a2958-6ff8-4f9c-a09d-09d1aea2a210 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:06.282758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:06.282758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:06.282758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:06.282758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:06.282758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:06.282758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:06.282758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:06.282758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:06.282758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:06.282758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:06.282758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:06.282758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:06.282758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:06.282758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:06.282758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:06.283985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:06.283985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:06.283985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:06.283985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:06.283985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:06.283985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:06.283985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:06.283985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:06.283985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:06.283985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:06.283985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:06.283985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:06.283985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:06.283985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:06.283985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:06.283985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:06.285550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:06.285550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:06.285550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:06.285550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:06.285550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:06.285550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:06.285550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:06.285550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:06.285550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:06.285550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:06.285550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:06.285550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:00:06.285550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 581/1149] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 15:00:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:06.290136 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7012a-d833-41e2-8fbf-27cb64e4a64d None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:06.392414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7012a-d833-41e2-8fbf-27cb64e4a64d None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:06.436175 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d104c92-1976-4563-827e-97920d7237bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:06.436827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d104c92-1976-4563-827e-97920d7237bd None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:06.437109 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d104c92-1976-4563-827e-97920d7237bd None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:06.437384 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d104c92-1976-4563-827e-97920d7237bd None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:06.438105 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d104c92-1976-4563-827e-97920d7237bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:06.438405 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d104c92-1976-4563-827e-97920d7237bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:06.440322 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d104c92-1976-4563-827e-97920d7237bd None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:06.449757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 569/1150] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 15:00:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:06.456523 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a2958-6ff8-4f9c-a09d-09d1aea2a210 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:06.569465 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a2958-6ff8-4f9c-a09d-09d1aea2a210 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:06.605098 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978786a-fde7-42df-94f9-8bc2b71aa469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:06.605733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978786a-fde7-42df-94f9-8bc2b71aa469 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:06.606015 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978786a-fde7-42df-94f9-8bc2b71aa469 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:06.606284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978786a-fde7-42df-94f9-8bc2b71aa469 None admin] PATH_INFO: `/v3/users/76137b3d6a664fa29caa1c4abfb9d211` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:06.607477 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978786a-fde7-42df-94f9-8bc2b71aa469 None admin] RBAC: Authorizing `identity:get_user(user_id=76137b3d6a664fa29caa1c4abfb9d211)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:06.608943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978786a-fde7-42df-94f9-8bc2b71aa469 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:06.610573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 582/1151] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 15:00:06 2022] GET /identity/v3/users/76137b3d6a664fa29caa1c4abfb9d211 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:06.616699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d104c92-1976-4563-827e-97920d7237bd None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:06.702591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:06.756423 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d104c92-1976-4563-827e-97920d7237bd None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:06.795093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59971862-57d8-40c9-837a-ff89c88bcd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:06.795650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59971862-57d8-40c9-837a-ff89c88bcd79 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:06.795921 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59971862-57d8-40c9-837a-ff89c88bcd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:06.796194 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59971862-57d8-40c9-837a-ff89c88bcd79 None admin] PATH_INFO: `/v3/projects/95369f2b03f94749a49041fc7e0a8f6b` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:06.797392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59971862-57d8-40c9-837a-ff89c88bcd79 None admin] RBAC: Authorizing `identity:get_project(project_id=95369f2b03f94749a49041fc7e0a8f6b)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:06.799315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59971862-57d8-40c9-837a-ff89c88bcd79 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:06.801035 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 570/1152] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 15:00:06 2022] GET /identity/v3/projects/95369f2b03f94749a49041fc7e0a8f6b => generated 311 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:06.807473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978786a-fde7-42df-94f9-8bc2b71aa469 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:06.866212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:06.919301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978786a-fde7-42df-94f9-8bc2b71aa469 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:06.956908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbf85f-a521-46ba-91e8-2021dd1f6fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:06.957567 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbf85f-a521-46ba-91e8-2021dd1f6fe5 None admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:06.957842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbf85f-a521-46ba-91e8-2021dd1f6fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:06.958116 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbf85f-a521-46ba-91e8-2021dd1f6fe5 None admin] PATH_INFO: `/v3/projects/95369f2b03f94749a49041fc7e0a8f6b/users/76137b3d6a664fa29caa1c4abfb9d211/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:06.960026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbf85f-a521-46ba-91e8-2021dd1f6fe5 None admin] RBAC: Authorizing `identity:create_grant(project_id=95369f2b03f94749a49041fc7e0a8f6b, user_id=76137b3d6a664fa29caa1c4abfb9d211, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:06.961474 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbf85f-a521-46ba-91e8-2021dd1f6fe5 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:06.982584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 583/1153] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 15:00:06 2022] PUT /identity/v3/projects/95369f2b03f94749a49041fc7e0a8f6b/users/76137b3d6a664fa29caa1c4abfb9d211/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:00:09.022651 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b8023-1486-49f4-9f05-49c087795a86 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:09.023460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b8023-1486-49f4-9f05-49c087795a86 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:09.023460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b8023-1486-49f4-9f05-49c087795a86 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:09.024669 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 571/1154] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:00:09.030646 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38340236-5a59-46c6-9215-b0addc0ca267 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:09.031118 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38340236-5a59-46c6-9215-b0addc0ca267 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:09.031226 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38340236-5a59-46c6-9215-b0addc0ca267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:09.050315 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-38340236-5a59-46c6-9215-b0addc0ca267 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:09.051554 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-38340236-5a59-46c6-9215-b0addc0ca267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:09.128812 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 584/1155] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:00:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:00:09.152035 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b8023-1486-49f4-9f05-49c087795a86 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:09.259328 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b8023-1486-49f4-9f05-49c087795a86 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:09.282758 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:09.300736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1b4d8-2d73-4d13-b201-f830d3505a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:09.301524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1b4d8-2d73-4d13-b201-f830d3505a4d None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:09.301808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1b4d8-2d73-4d13-b201-f830d3505a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:09.302078 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1b4d8-2d73-4d13-b201-f830d3505a4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:09.312271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1b4d8-2d73-4d13-b201-f830d3505a4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:09.317077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-16e1b4d8-2d73-4d13-b201-f830d3505a4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:09.317077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:09.317077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:09.317077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:09.317077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:09.317077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:09.317077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:09.317077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:09.317077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:09.317077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:09.317077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:09.317077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:09.317077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:09.317077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:09.317077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:09.317077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:09.318276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:09.318276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:09.318276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:09.318276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:09.318276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:09.318276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:09.318276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:09.318276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:09.318276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:09.318276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:09.318276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:09.318276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:09.318276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:09.318276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:09.318276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:09.318276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:09.319380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:09.319380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:09.319380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:09.319380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:09.319380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:09.319380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:09.319380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:09.319380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:09.319380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:09.319380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:09.319380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:09.319380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:00:09.319380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 572/1156] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 15:00:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:09.324637 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38340236-5a59-46c6-9215-b0addc0ca267 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:09.373754 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:09.436236 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38340236-5a59-46c6-9215-b0addc0ca267 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:09.471536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48321b5-25a8-48e8-a8ef-5d1c9d57d8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:09.472557 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48321b5-25a8-48e8-a8ef-5d1c9d57d8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:09.472882 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48321b5-25a8-48e8-a8ef-5d1c9d57d8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:09.473205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48321b5-25a8-48e8-a8ef-5d1c9d57d8c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:09.473996 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48321b5-25a8-48e8-a8ef-5d1c9d57d8c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:09.474347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48321b5-25a8-48e8-a8ef-5d1c9d57d8c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:09.475837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48321b5-25a8-48e8-a8ef-5d1c9d57d8c5 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:09.485476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 585/1157] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 15:00:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:09.492693 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1b4d8-2d73-4d13-b201-f830d3505a4d None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:09.602028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1b4d8-2d73-4d13-b201-f830d3505a4d None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:09.639943 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db9aa6-df68-4358-97aa-88d1323974f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:09.640602 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db9aa6-df68-4358-97aa-88d1323974f7 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:09.640879 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db9aa6-df68-4358-97aa-88d1323974f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:09.641149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db9aa6-df68-4358-97aa-88d1323974f7 None admin] PATH_INFO: `/v3/users/76137b3d6a664fa29caa1c4abfb9d211` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:09.642820 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db9aa6-df68-4358-97aa-88d1323974f7 None admin] RBAC: Authorizing `identity:get_user(user_id=76137b3d6a664fa29caa1c4abfb9d211)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:09.644287 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db9aa6-df68-4358-97aa-88d1323974f7 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:09.645980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 573/1158] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 15:00:09 2022] GET /identity/v3/users/76137b3d6a664fa29caa1c4abfb9d211 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:09.653187 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48321b5-25a8-48e8-a8ef-5d1c9d57d8c5 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:09.802014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48321b5-25a8-48e8-a8ef-5d1c9d57d8c5 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:09.811058 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:09.837688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5b040-08f5-4de8-9857-222353b6d2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:09.838302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5b040-08f5-4de8-9857-222353b6d2fa None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:09.838578 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5b040-08f5-4de8-9857-222353b6d2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:09.838853 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5b040-08f5-4de8-9857-222353b6d2fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:09.915123 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db5b040-08f5-4de8-9857-222353b6d2fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:09.916506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db5b040-08f5-4de8-9857-222353b6d2fa None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:10.021331 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 586/1159] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 15:00:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:00:10.027373 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db9aa6-df68-4358-97aa-88d1323974f7 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:10.083790 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:10.158779 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db9aa6-df68-4358-97aa-88d1323974f7 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:10.193238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1135a89-4b34-46b2-a57f-a565d2c77e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:10.193787 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1135a89-4b34-46b2-a57f-a565d2c77e8e None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:10.194064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1135a89-4b34-46b2-a57f-a565d2c77e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:10.194334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1135a89-4b34-46b2-a57f-a565d2c77e8e None admin] PATH_INFO: `/v3/projects/95369f2b03f94749a49041fc7e0a8f6b` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:10.195481 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1135a89-4b34-46b2-a57f-a565d2c77e8e None admin] RBAC: Authorizing `identity:get_project(project_id=95369f2b03f94749a49041fc7e0a8f6b)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:10.196948 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1135a89-4b34-46b2-a57f-a565d2c77e8e None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:10.198610 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 574/1160] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 15:00:10 2022] GET /identity/v3/projects/95369f2b03f94749a49041fc7e0a8f6b => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:10.205239 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5b040-08f5-4de8-9857-222353b6d2fa None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:10.216326 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:10.386226 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5b040-08f5-4de8-9857-222353b6d2fa None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:10.429870 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a51cfd-fdf8-451d-9aad-0010273c39b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:10.430511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a51cfd-fdf8-451d-9aad-0010273c39b4 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:10.430791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a51cfd-fdf8-451d-9aad-0010273c39b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:10.431764 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a51cfd-fdf8-451d-9aad-0010273c39b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:10.431918 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a51cfd-fdf8-451d-9aad-0010273c39b4 None admin] RBAC: Adding query filter params (role.id=5456887a306649bf80a9ea7bd4282e97, scope.project.id=95369f2b03f94749a49041fc7e0a8f6b, user.id=76137b3d6a664fa29caa1c4abfb9d211) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:10.432194 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a51cfd-fdf8-451d-9aad-0010273c39b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:10.433686 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a51cfd-fdf8-451d-9aad-0010273c39b4 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:10.440411 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 587/1161] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 15:00:10 2022] GET /identity/v3/role_assignments?user.id=76137b3d6a664fa29caa1c4abfb9d211&scope.project.id=95369f2b03f94749a49041fc7e0a8f6b&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:12.561046 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc7df9-bdce-456c-b612-4aade030af35 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:12.561737 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc7df9-bdce-456c-b612-4aade030af35 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:12.561737 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc7df9-bdce-456c-b612-4aade030af35 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:12.562918 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 575/1162] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:00:12.569813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849491c-d4d4-4212-aad3-fbb349a1c72a None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:12.570349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849491c-d4d4-4212-aad3-fbb349a1c72a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:12.570444 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849491c-d4d4-4212-aad3-fbb349a1c72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:12.594868 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-1849491c-d4d4-4212-aad3-fbb349a1c72a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:12.596602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-1849491c-d4d4-4212-aad3-fbb349a1c72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:12.683905 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 588/1163] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:00:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:00:12.711332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc7df9-bdce-456c-b612-4aade030af35 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:12.825749 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc7df9-bdce-456c-b612-4aade030af35 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:12.845076 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:12.864582 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24736d8f-5b80-4146-a3fd-fc6444cb2414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:12.865245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24736d8f-5b80-4146-a3fd-fc6444cb2414 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:12.865560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24736d8f-5b80-4146-a3fd-fc6444cb2414 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:12.865757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24736d8f-5b80-4146-a3fd-fc6444cb2414 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:12.941308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24736d8f-5b80-4146-a3fd-fc6444cb2414 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:12.942809 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24736d8f-5b80-4146-a3fd-fc6444cb2414 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:13.053945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 576/1164] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 15:00:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:00:13.060009 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849491c-d4d4-4212-aad3-fbb349a1c72a None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:13.062242 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:13.170765 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849491c-d4d4-4212-aad3-fbb349a1c72a None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:13.207704 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383539f9-5eea-4839-bb9a-f273af381c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:13.208382 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383539f9-5eea-4839-bb9a-f273af381c62 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:13.208689 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383539f9-5eea-4839-bb9a-f273af381c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:13.208968 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383539f9-5eea-4839-bb9a-f273af381c62 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:13.216384 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383539f9-5eea-4839-bb9a-f273af381c62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:13.217879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383539f9-5eea-4839-bb9a-f273af381c62 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:13.230229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-383539f9-5eea-4839-bb9a-f273af381c62 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 15:00:13.230229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:13.230229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 01 15:00:13.230229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 01 15:00:13.230229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:00:13.230229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:00:13.230229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:00:13.230229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:00:13.230229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 01 15:00:13.230229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:00:13.230229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:00:13.230229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:00:13.230229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:13.230229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:13.230229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:13.230229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:13.230229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:13.231540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:13.231540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:13.231540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:13.231540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:13.231540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:13.231540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:13.231540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:13.231540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:13.231540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:13.231540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:13.231540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:13.231540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:13.231540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:13.231540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:13.231540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:13.231540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:13.232485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:13.232485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:13.232485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:13.232485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:13.232485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:13.232485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:13.232485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:13.232485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:13.232485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:13.232485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:13.232485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 01 15:00:13.232485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:13.232485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 15:00:13.232485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:00:13.232485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 589/1165] 10.222.0.93 () {56 vars in 1117 bytes} [Mon Aug 1 15:00:13 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:13.238047 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24736d8f-5b80-4146-a3fd-fc6444cb2414 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:13.258092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:13.354803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24736d8f-5b80-4146-a3fd-fc6444cb2414 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:13.409569 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f0efd-778f-4c74-982b-20737ac11ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:13.410180 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f0efd-778f-4c74-982b-20737ac11ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:13.410459 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f0efd-778f-4c74-982b-20737ac11ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:13.410764 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f0efd-778f-4c74-982b-20737ac11ca3 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:13.411450 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f0efd-778f-4c74-982b-20737ac11ca3 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:13.411780 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f0efd-778f-4c74-982b-20737ac11ca3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:13.413145 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f0efd-778f-4c74-982b-20737ac11ca3 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:13.420724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 577/1166] 10.222.0.93 () {56 vars in 1115 bytes} [Mon Aug 1 15:00:13 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:15.553521 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9918d4-9f42-4944-9dfc-4ca16b206bc9 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:15.554055 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9918d4-9f42-4944-9dfc-4ca16b206bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:15.554055 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9918d4-9f42-4944-9dfc-4ca16b206bc9 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:15.555412 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 590/1167] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:00:15.560656 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a702a52-5222-42c4-a7b3-8f4b6a74dd30 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:15.561076 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a702a52-5222-42c4-a7b3-8f4b6a74dd30 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:15.561215 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a702a52-5222-42c4-a7b3-8f4b6a74dd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:15.579074 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-3a702a52-5222-42c4-a7b3-8f4b6a74dd30 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:15.580308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-3a702a52-5222-42c4-a7b3-8f4b6a74dd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:15.650148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 578/1168] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:00:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:00:15.673807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9918d4-9f42-4944-9dfc-4ca16b206bc9 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:15.737857 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:15.779508 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9918d4-9f42-4944-9dfc-4ca16b206bc9 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:15.813164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478242b-1999-41c8-808c-fd7d999ebfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:15.813780 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478242b-1999-41c8-808c-fd7d999ebfe0 None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:15.814090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478242b-1999-41c8-808c-fd7d999ebfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:15.814370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478242b-1999-41c8-808c-fd7d999ebfe0 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:15.815071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478242b-1999-41c8-808c-fd7d999ebfe0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:15.816446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478242b-1999-41c8-808c-fd7d999ebfe0 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:15.833119 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 591/1169] 10.222.0.93 () {60 vars in 1138 bytes} [Mon Aug 1 15:00:15 2022] POST /identity/v3/domains => generated 267 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:00:17.892557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a94def-93e3-429d-9c39-76606fae5a2c None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:17.893121 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a94def-93e3-429d-9c39-76606fae5a2c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:17.893121 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a94def-93e3-429d-9c39-76606fae5a2c None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:17.894557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 579/1170] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:00:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:00:17.900030 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5c74b-b9a0-4cf5-8e88-9f1babc71ef2 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:17.900512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5c74b-b9a0-4cf5-8e88-9f1babc71ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:17.900615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5c74b-b9a0-4cf5-8e88-9f1babc71ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:17.918833 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-8af5c74b-b9a0-4cf5-8e88-9f1babc71ef2 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:17.920040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-8af5c74b-b9a0-4cf5-8e88-9f1babc71ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:17.990672 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 592/1171] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:00:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:00:18.015328 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a94def-93e3-429d-9c39-76606fae5a2c None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:18.030550 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:18.128054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a94def-93e3-429d-9c39-76606fae5a2c None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:18.162727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dcd83e-a980-4180-b57b-60ca3aa6a1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:18.163297 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dcd83e-a980-4180-b57b-60ca3aa6a1fb None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:18.163582 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dcd83e-a980-4180-b57b-60ca3aa6a1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:18.163859 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dcd83e-a980-4180-b57b-60ca3aa6a1fb None admin] PATH_INFO: `/v3/domains/773cfa6f1f694df38c1cc6caeab5da28` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:18.175609 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dcd83e-a980-4180-b57b-60ca3aa6a1fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=773cfa6f1f694df38c1cc6caeab5da28)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:18.177141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dcd83e-a980-4180-b57b-60ca3aa6a1fb None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:18.178863 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 580/1172] 10.222.0.93 () {56 vars in 1183 bytes} [Mon Aug 1 15:00:18 2022] GET /identity/v3/domains/773cfa6f1f694df38c1cc6caeab5da28 => generated 267 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:18.185055 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5c74b-b9a0-4cf5-8e88-9f1babc71ef2 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:18.242044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:18.296885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5c74b-b9a0-4cf5-8e88-9f1babc71ef2 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:18.331687 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ed0fd-7a48-4158-869a-19def2bd60c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:18.332312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ed0fd-7a48-4158-869a-19def2bd60c2 None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:18.332621 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ed0fd-7a48-4158-869a-19def2bd60c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:18.332897 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ed0fd-7a48-4158-869a-19def2bd60c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:18.333578 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ed0fd-7a48-4158-869a-19def2bd60c2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:18.335020 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ed0fd-7a48-4158-869a-19def2bd60c2 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:18.365007 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 593/1173] 10.222.0.93 () {60 vars in 1141 bytes} [Mon Aug 1 15:00:18 2022] POST /identity/v3/projects => generated 361 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:00:20.510875 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eacb1f-94d2-4ecc-bba1-61edf50b451f None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:20.511323 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eacb1f-94d2-4ecc-bba1-61edf50b451f None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:20.511418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eacb1f-94d2-4ecc-bba1-61edf50b451f None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:20.512867 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 581/1174] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:00:20.518491 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a507442-a10a-4f15-9bd8-b94492ab84ec None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:20.519060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a507442-a10a-4f15-9bd8-b94492ab84ec None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:20.519060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a507442-a10a-4f15-9bd8-b94492ab84ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:20.539016 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-5a507442-a10a-4f15-9bd8-b94492ab84ec None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:20.540373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5a507442-a10a-4f15-9bd8-b94492ab84ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:20.609521 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 594/1175] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:00:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:00:20.635567 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eacb1f-94d2-4ecc-bba1-61edf50b451f None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:20.712365 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:20.745687 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eacb1f-94d2-4ecc-bba1-61edf50b451f None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:20.787025 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9ce15-66f5-41cc-bbbf-d8c1d6942219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:20.787576 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9ce15-66f5-41cc-bbbf-d8c1d6942219 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:20.787856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9ce15-66f5-41cc-bbbf-d8c1d6942219 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:20.788130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9ce15-66f5-41cc-bbbf-d8c1d6942219 None admin] PATH_INFO: `/v3/domains/773cfa6f1f694df38c1cc6caeab5da28` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:20.789316 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9ce15-66f5-41cc-bbbf-d8c1d6942219 None admin] RBAC: Authorizing `identity:get_domain(domain_id=773cfa6f1f694df38c1cc6caeab5da28)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:20.790757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9ce15-66f5-41cc-bbbf-d8c1d6942219 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:20.792377 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 582/1176] 10.222.0.93 () {56 vars in 1183 bytes} [Mon Aug 1 15:00:20 2022] GET /identity/v3/domains/773cfa6f1f694df38c1cc6caeab5da28 => generated 267 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:20.799106 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a507442-a10a-4f15-9bd8-b94492ab84ec None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:20.832947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:20.909013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a507442-a10a-4f15-9bd8-b94492ab84ec None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:20.953284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c70fb2-29fc-4b65-a7ab-f1e45d603ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:20.953900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c70fb2-29fc-4b65-a7ab-f1e45d603ed1 None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:20.954181 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c70fb2-29fc-4b65-a7ab-f1e45d603ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:20.954989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c70fb2-29fc-4b65-a7ab-f1e45d603ed1 None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:20.955696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c70fb2-29fc-4b65-a7ab-f1e45d603ed1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:20.957181 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c70fb2-29fc-4b65-a7ab-f1e45d603ed1 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:20.980403 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 595/1177] 10.222.0.93 () {60 vars in 1132 bytes} [Mon Aug 1 15:00:20 2022] POST /identity/v3/users => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:00:23.084134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d455b-cb14-4904-850b-311a8c8f0daa None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:23.084991 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d455b-cb14-4904-850b-311a8c8f0daa None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:23.084991 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d455b-cb14-4904-850b-311a8c8f0daa None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:23.086188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 583/1178] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:00:23.092910 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8101a2-0bda-4c0c-8b9a-1ecacd80102d None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:23.093468 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8101a2-0bda-4c0c-8b9a-1ecacd80102d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:23.093468 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8101a2-0bda-4c0c-8b9a-1ecacd80102d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:23.114893 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-3e8101a2-0bda-4c0c-8b9a-1ecacd80102d None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:23.116117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3e8101a2-0bda-4c0c-8b9a-1ecacd80102d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:23.191307 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 596/1179] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:00:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:00:23.218567 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d455b-cb14-4904-850b-311a8c8f0daa None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:23.327578 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d455b-cb14-4904-850b-311a8c8f0daa None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:23.363426 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b79cb-86dd-4b61-bf1d-3b2cb5e20646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:23.363974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b79cb-86dd-4b61-bf1d-3b2cb5e20646 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:23.364249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b79cb-86dd-4b61-bf1d-3b2cb5e20646 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:23.364555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b79cb-86dd-4b61-bf1d-3b2cb5e20646 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:23.374005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:23.377910 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b79cb-86dd-4b61-bf1d-3b2cb5e20646 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:23.383041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-355b79cb-86dd-4b61-bf1d-3b2cb5e20646 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:23.383041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:23.383041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:23.383041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:23.383041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:23.383041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:23.383041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:23.383041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:23.383041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:23.383041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:23.383041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:23.383041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:23.383041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:23.383041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:23.383041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:23.383041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:23.384291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:23.384291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:23.384291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:23.384291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:23.384291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:23.384291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:23.384291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:23.384291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:23.384291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:23.384291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:23.384291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:23.384291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:23.384291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:23.384291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:23.384291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:23.384291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:23.385560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:23.385560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:23.385560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:23.385560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:23.385560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:23.385560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:23.385560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:23.385560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:23.385560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:23.385560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:23.385560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:23.385560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:00:23.385560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 584/1180] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 15:00:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:23.390612 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8101a2-0bda-4c0c-8b9a-1ecacd80102d None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:23.414174 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:23.505680 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8101a2-0bda-4c0c-8b9a-1ecacd80102d None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:23.547345 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749ef25-7ef3-40a7-b667-cf2829ccab06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:23.548078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749ef25-7ef3-40a7-b667-cf2829ccab06 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:23.548428 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749ef25-7ef3-40a7-b667-cf2829ccab06 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:23.548778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749ef25-7ef3-40a7-b667-cf2829ccab06 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:23.549553 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749ef25-7ef3-40a7-b667-cf2829ccab06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:23.549902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749ef25-7ef3-40a7-b667-cf2829ccab06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:23.551339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749ef25-7ef3-40a7-b667-cf2829ccab06 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:23.560060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 597/1181] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 15:00:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:23.566546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b79cb-86dd-4b61-bf1d-3b2cb5e20646 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:23.677360 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b79cb-86dd-4b61-bf1d-3b2cb5e20646 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:23.714813 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b4ef5-35db-465d-94fc-a115d6170b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:23.715443 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b4ef5-35db-465d-94fc-a115d6170b86 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:23.715726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b4ef5-35db-465d-94fc-a115d6170b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:23.716002 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b4ef5-35db-465d-94fc-a115d6170b86 None admin] PATH_INFO: `/v3/users/878e1107cd1f49668dd3ed6a00b91a5b` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:23.729699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b4ef5-35db-465d-94fc-a115d6170b86 None admin] RBAC: Authorizing `identity:get_user(user_id=878e1107cd1f49668dd3ed6a00b91a5b)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:23.731164 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b4ef5-35db-465d-94fc-a115d6170b86 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:23.732902 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 585/1182] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 15:00:23 2022] GET /identity/v3/users/878e1107cd1f49668dd3ed6a00b91a5b => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:23.738964 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749ef25-7ef3-40a7-b667-cf2829ccab06 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:23.824853 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:23.843860 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749ef25-7ef3-40a7-b667-cf2829ccab06 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:23.879212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086a022-9a33-473a-82a5-d676393132dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:23.879892 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086a022-9a33-473a-82a5-d676393132dc None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:23.880221 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086a022-9a33-473a-82a5-d676393132dc None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:23.880611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086a022-9a33-473a-82a5-d676393132dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:23.956192 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086a022-9a33-473a-82a5-d676393132dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:23.957609 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086a022-9a33-473a-82a5-d676393132dc None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:24.047821 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:24.071010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 598/1183] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 15:00:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:00:24.079658 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b4ef5-35db-465d-94fc-a115d6170b86 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:24.143557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:24.191361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b4ef5-35db-465d-94fc-a115d6170b86 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:24.228748 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdb8d3-37fd-463f-a41a-8777a7416123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:24.229321 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdb8d3-37fd-463f-a41a-8777a7416123 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:24.229591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdb8d3-37fd-463f-a41a-8777a7416123 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:24.229864 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdb8d3-37fd-463f-a41a-8777a7416123 None admin] PATH_INFO: `/v3/projects/4a03236356e34847a05c9ecf7471a2fb` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:24.231085 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdb8d3-37fd-463f-a41a-8777a7416123 None admin] RBAC: Authorizing `identity:get_project(project_id=4a03236356e34847a05c9ecf7471a2fb)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:24.232584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdb8d3-37fd-463f-a41a-8777a7416123 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:24.234343 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 586/1184] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 15:00:24 2022] GET /identity/v3/projects/4a03236356e34847a05c9ecf7471a2fb => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:24.241421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086a022-9a33-473a-82a5-d676393132dc None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:24.351138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086a022-9a33-473a-82a5-d676393132dc None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:24.386394 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61836df5-ac13-4ee4-8976-076be4b7030f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:24.387004 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61836df5-ac13-4ee4-8976-076be4b7030f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:24.387277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61836df5-ac13-4ee4-8976-076be4b7030f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:24.387545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61836df5-ac13-4ee4-8976-076be4b7030f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:24.388313 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61836df5-ac13-4ee4-8976-076be4b7030f None admin] RBAC: Adding query filter params (role.id=5456887a306649bf80a9ea7bd4282e97, scope.project.id=4a03236356e34847a05c9ecf7471a2fb, user.id=878e1107cd1f49668dd3ed6a00b91a5b) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:24.388641 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61836df5-ac13-4ee4-8976-076be4b7030f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:24.390077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61836df5-ac13-4ee4-8976-076be4b7030f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:24.395662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 599/1185] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 15:00:24 2022] GET /identity/v3/role_assignments?user.id=878e1107cd1f49668dd3ed6a00b91a5b&scope.project.id=4a03236356e34847a05c9ecf7471a2fb&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:26.406041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466cc1b-cb1b-4f23-bdcd-08b68fe9d44a None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:26.406590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466cc1b-cb1b-4f23-bdcd-08b68fe9d44a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:26.406702 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466cc1b-cb1b-4f23-bdcd-08b68fe9d44a None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:26.408802 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 587/1186] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:00:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:00:26.414750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39535611-bf52-4cba-b2ea-5631c2f2bf7d None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:26.415243 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39535611-bf52-4cba-b2ea-5631c2f2bf7d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:26.415356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39535611-bf52-4cba-b2ea-5631c2f2bf7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:26.435031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-39535611-bf52-4cba-b2ea-5631c2f2bf7d None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:26.436230 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-39535611-bf52-4cba-b2ea-5631c2f2bf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:26.487467 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:26.519625 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 600/1187] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:00:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:00:26.542791 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466cc1b-cb1b-4f23-bdcd-08b68fe9d44a None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:26.665145 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466cc1b-cb1b-4f23-bdcd-08b68fe9d44a None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:26.696715 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:26.706845 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b0189-46cf-4189-985b-034531a54f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:26.707483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b0189-46cf-4189-985b-034531a54f47 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:26.707763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b0189-46cf-4189-985b-034531a54f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:26.708037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b0189-46cf-4189-985b-034531a54f47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:26.719495 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b0189-46cf-4189-985b-034531a54f47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:26.724801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-0c6b0189-46cf-4189-985b-034531a54f47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:26.724801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:26.724801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:26.724801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:26.724801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:26.724801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:26.724801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:26.724801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:26.724801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:26.724801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:26.724801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:26.724801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:26.724801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:26.724801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:26.724801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:26.724801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:26.726393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:26.726393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:26.726393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:26.726393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:26.726393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:26.726393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:26.726393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:26.726393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:26.726393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:26.726393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:26.726393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:26.726393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:26.726393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:26.726393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:26.726393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:26.726393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:26.727285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:26.727285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:26.727285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:26.727285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:26.727285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:26.727285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:26.727285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:26.727285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:26.727285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:26.727285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:26.727285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:26.727285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:00:26.727285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 588/1188] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 15:00:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:26.733666 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39535611-bf52-4cba-b2ea-5631c2f2bf7d None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:26.839316 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39535611-bf52-4cba-b2ea-5631c2f2bf7d None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:26.875689 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd1763-6880-43c4-b8f4-d5de5b47ae1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:26.876377 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd1763-6880-43c4-b8f4-d5de5b47ae1c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:26.876681 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd1763-6880-43c4-b8f4-d5de5b47ae1c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:26.876952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd1763-6880-43c4-b8f4-d5de5b47ae1c None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:26.877675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd1763-6880-43c4-b8f4-d5de5b47ae1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:26.877967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd1763-6880-43c4-b8f4-d5de5b47ae1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:26.879353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd1763-6880-43c4-b8f4-d5de5b47ae1c None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:26.888259 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 601/1189] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 15:00:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:26.894762 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b0189-46cf-4189-985b-034531a54f47 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:27.002567 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b0189-46cf-4189-985b-034531a54f47 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:27.041537 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cb41d-d551-437f-bf0a-031d9c626244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:27.042182 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cb41d-d551-437f-bf0a-031d9c626244 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:27.042469 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cb41d-d551-437f-bf0a-031d9c626244 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:27.042793 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cb41d-d551-437f-bf0a-031d9c626244 None admin] PATH_INFO: `/v3/users/878e1107cd1f49668dd3ed6a00b91a5b` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:27.044026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cb41d-d551-437f-bf0a-031d9c626244 None admin] RBAC: Authorizing `identity:get_user(user_id=878e1107cd1f49668dd3ed6a00b91a5b)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:27.045522 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cb41d-d551-437f-bf0a-031d9c626244 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:27.047192 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 589/1190] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 15:00:26 2022] GET /identity/v3/users/878e1107cd1f49668dd3ed6a00b91a5b => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:27.053492 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd1763-6880-43c4-b8f4-d5de5b47ae1c None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:27.099352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:27.161525 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd1763-6880-43c4-b8f4-d5de5b47ae1c None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:27.195680 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0abab6-43b0-48cb-81e9-907a3a94ee4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:27.196309 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0abab6-43b0-48cb-81e9-907a3a94ee4e None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:27.196621 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0abab6-43b0-48cb-81e9-907a3a94ee4e None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:27.196900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0abab6-43b0-48cb-81e9-907a3a94ee4e None admin] PATH_INFO: `/v3/projects/4a03236356e34847a05c9ecf7471a2fb` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:27.198058 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0abab6-43b0-48cb-81e9-907a3a94ee4e None admin] RBAC: Authorizing `identity:get_project(project_id=4a03236356e34847a05c9ecf7471a2fb)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:27.199616 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0abab6-43b0-48cb-81e9-907a3a94ee4e None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:27.201342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 602/1191] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 15:00:27 2022] GET /identity/v3/projects/4a03236356e34847a05c9ecf7471a2fb => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:27.207954 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cb41d-d551-437f-bf0a-031d9c626244 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:27.263936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:27.309123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cb41d-d551-437f-bf0a-031d9c626244 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:27.345089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c6fde-42aa-49a7-b52b-456efcb7f3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:27.345744 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c6fde-42aa-49a7-b52b-456efcb7f3dc None admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:27.346021 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c6fde-42aa-49a7-b52b-456efcb7f3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:27.346293 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c6fde-42aa-49a7-b52b-456efcb7f3dc None admin] PATH_INFO: `/v3/projects/4a03236356e34847a05c9ecf7471a2fb/users/878e1107cd1f49668dd3ed6a00b91a5b/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:27.348207 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c6fde-42aa-49a7-b52b-456efcb7f3dc None admin] RBAC: Authorizing `identity:create_grant(project_id=4a03236356e34847a05c9ecf7471a2fb, user_id=878e1107cd1f49668dd3ed6a00b91a5b, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:27.349657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c6fde-42aa-49a7-b52b-456efcb7f3dc None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:27.362720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 590/1192] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 15:00:27 2022] PUT /identity/v3/projects/4a03236356e34847a05c9ecf7471a2fb/users/878e1107cd1f49668dd3ed6a00b91a5b/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:00:29.379859 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280826fd-0b41-437f-bedc-15bf36c07a0d None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:29.380440 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280826fd-0b41-437f-bedc-15bf36c07a0d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:29.380440 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280826fd-0b41-437f-bedc-15bf36c07a0d None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:29.381834 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 603/1193] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:00:29.387134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b281d-46d7-4851-a9f4-311c4f02596c None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:29.387562 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b281d-46d7-4851-a9f4-311c4f02596c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:29.387695 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b281d-46d7-4851-a9f4-311c4f02596c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:29.406531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-e68b281d-46d7-4851-a9f4-311c4f02596c None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:29.407775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e68b281d-46d7-4851-a9f4-311c4f02596c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:29.475222 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 591/1194] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:00:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:00:29.499772 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280826fd-0b41-437f-bedc-15bf36c07a0d None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:29.605477 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280826fd-0b41-437f-bedc-15bf36c07a0d None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:29.613235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:29.642783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8b9f3-9883-46a1-8f97-0d33abae87b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:29.643322 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8b9f3-9883-46a1-8f97-0d33abae87b7 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:29.643610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8b9f3-9883-46a1-8f97-0d33abae87b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:29.643881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8b9f3-9883-46a1-8f97-0d33abae87b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:29.654281 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8b9f3-9883-46a1-8f97-0d33abae87b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:29.660134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-40f8b9f3-9883-46a1-8f97-0d33abae87b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:29.660134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:29.660134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:29.660134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:29.660134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:29.660134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:29.660134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:29.660134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:29.660134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:29.660134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:29.660134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:29.660134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:29.660134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:29.660134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:29.660134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:29.660134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:29.661483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:29.661483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:29.661483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:29.661483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:29.661483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:29.661483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:29.661483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:29.661483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:29.661483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:29.661483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:29.661483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:29.661483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:29.661483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:29.661483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:29.661483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:29.661483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:29.663421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:29.663421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:29.663421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:29.663421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:29.663421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:29.663421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:29.663421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:29.663421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:29.663421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:29.663421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:29.663421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:29.663421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:00:29.663421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 604/1195] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 15:00:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:29.667732 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b281d-46d7-4851-a9f4-311c4f02596c None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:29.721855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:29.781876 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b281d-46d7-4851-a9f4-311c4f02596c None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:29.815998 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c8a3a-6bdb-4bf2-9101-842bcec564f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:29.816643 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c8a3a-6bdb-4bf2-9101-842bcec564f2 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:29.816926 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c8a3a-6bdb-4bf2-9101-842bcec564f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:29.817198 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c8a3a-6bdb-4bf2-9101-842bcec564f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:29.817909 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c8a3a-6bdb-4bf2-9101-842bcec564f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:29.818209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c8a3a-6bdb-4bf2-9101-842bcec564f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:29.819580 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c8a3a-6bdb-4bf2-9101-842bcec564f2 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:29.828867 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 592/1196] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 15:00:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:29.834924 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8b9f3-9883-46a1-8f97-0d33abae87b7 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:29.940029 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8b9f3-9883-46a1-8f97-0d33abae87b7 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:29.976353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1b543-fbeb-4d16-a3a8-160c2ddb2922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:29.977036 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1b543-fbeb-4d16-a3a8-160c2ddb2922 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:29.977316 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1b543-fbeb-4d16-a3a8-160c2ddb2922 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:29.977592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1b543-fbeb-4d16-a3a8-160c2ddb2922 None admin] PATH_INFO: `/v3/users/878e1107cd1f49668dd3ed6a00b91a5b` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:29.978776 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1b543-fbeb-4d16-a3a8-160c2ddb2922 None admin] RBAC: Authorizing `identity:get_user(user_id=878e1107cd1f49668dd3ed6a00b91a5b)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:29.980211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1b543-fbeb-4d16-a3a8-160c2ddb2922 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:29.981876 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 605/1197] 10.222.0.93 () {56 vars in 1177 bytes} [Mon Aug 1 15:00:29 2022] GET /identity/v3/users/878e1107cd1f49668dd3ed6a00b91a5b => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:29.988401 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c8a3a-6bdb-4bf2-9101-842bcec564f2 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:30.102183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c8a3a-6bdb-4bf2-9101-842bcec564f2 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:30.109710 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:30.138909 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341018ec-5723-4cfe-9f23-a1576006b37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:30.139536 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341018ec-5723-4cfe-9f23-a1576006b37a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:30.139847 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341018ec-5723-4cfe-9f23-a1576006b37a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:30.140190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341018ec-5723-4cfe-9f23-a1576006b37a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:30.213749 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341018ec-5723-4cfe-9f23-a1576006b37a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:30.215113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341018ec-5723-4cfe-9f23-a1576006b37a None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:30.318606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 593/1198] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 15:00:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:00:30.324833 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1b543-fbeb-4d16-a3a8-160c2ddb2922 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:30.361896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:30.432168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1b543-fbeb-4d16-a3a8-160c2ddb2922 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:30.467624 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a591acf-8e97-4fd5-bec5-02aa23b800ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:30.468166 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a591acf-8e97-4fd5-bec5-02aa23b800ee None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:30.468439 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a591acf-8e97-4fd5-bec5-02aa23b800ee None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:30.468741 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a591acf-8e97-4fd5-bec5-02aa23b800ee None admin] PATH_INFO: `/v3/projects/4a03236356e34847a05c9ecf7471a2fb` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:30.469921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a591acf-8e97-4fd5-bec5-02aa23b800ee None admin] RBAC: Authorizing `identity:get_project(project_id=4a03236356e34847a05c9ecf7471a2fb)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:30.471362 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a591acf-8e97-4fd5-bec5-02aa23b800ee None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:30.473082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 606/1199] 10.222.0.93 () {56 vars in 1186 bytes} [Mon Aug 1 15:00:30 2022] GET /identity/v3/projects/4a03236356e34847a05c9ecf7471a2fb => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:30.479479 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341018ec-5723-4cfe-9f23-a1576006b37a None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:30.487659 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:30.580669 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341018ec-5723-4cfe-9f23-a1576006b37a None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:30.612942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ba4f1-dc5d-4e9d-80da-0b8013b3d99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:30.613543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ba4f1-dc5d-4e9d-80da-0b8013b3d99e None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:30.613824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ba4f1-dc5d-4e9d-80da-0b8013b3d99e None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:30.614098 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ba4f1-dc5d-4e9d-80da-0b8013b3d99e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:30.614870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ba4f1-dc5d-4e9d-80da-0b8013b3d99e None admin] RBAC: Adding query filter params (role.id=5456887a306649bf80a9ea7bd4282e97, scope.project.id=4a03236356e34847a05c9ecf7471a2fb, user.id=878e1107cd1f49668dd3ed6a00b91a5b) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:30.615178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ba4f1-dc5d-4e9d-80da-0b8013b3d99e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:30.616638 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ba4f1-dc5d-4e9d-80da-0b8013b3d99e None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:30.622650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 594/1200] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 15:00:30 2022] GET /identity/v3/role_assignments?user.id=878e1107cd1f49668dd3ed6a00b91a5b&scope.project.id=4a03236356e34847a05c9ecf7471a2fb&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:40.463370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=49 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:40.620012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=65 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:27.423349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd52c7c-29b3-45b6-8f52-e9db70298b9c None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:27.423946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd52c7c-29b3-45b6-8f52-e9db70298b9c None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:27.423946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd52c7c-29b3-45b6-8f52-e9db70298b9c None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:27.425629 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 607/1201] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:01:27.434985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23710667-eb4f-4b01-ae58-f0856b816811 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:27.434985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23710667-eb4f-4b01-ae58-f0856b816811 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:27.434985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23710667-eb4f-4b01-ae58-f0856b816811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:27.457733 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-23710667-eb4f-4b01-ae58-f0856b816811 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:27.459097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-23710667-eb4f-4b01-ae58-f0856b816811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:27.567340 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 595/1202] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:01:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:27.598914 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd52c7c-29b3-45b6-8f52-e9db70298b9c None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:27.759134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd52c7c-29b3-45b6-8f52-e9db70298b9c None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:27.832370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533c9c9-d0e9-43be-a89b-433d87decd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:27.833136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533c9c9-d0e9-43be-a89b-433d87decd7e None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:27.833440 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533c9c9-d0e9-43be-a89b-433d87decd7e None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:27.833764 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533c9c9-d0e9-43be-a89b-433d87decd7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:27.853688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8533c9c9-d0e9-43be-a89b-433d87decd7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:27.855381 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8533c9c9-d0e9-43be-a89b-433d87decd7e None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:27.876220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-8533c9c9-d0e9-43be-a89b-433d87decd7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:27.876220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:27.876220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:27.876220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:27.876220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:27.876220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:27.876220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:27.876220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:27.876220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:27.876220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:27.876220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:27.876220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:27.876220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:27.876220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:27.876220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:27.878303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:27.878303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:27.878303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:27.878303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:27.878303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:27.878303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:27.878303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:27.878303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:27.878303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:27.878303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:27.878303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:27.878303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:27.878303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:27.878303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:27.878303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:27.878303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:27.878303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:27.880214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:27.880214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:27.880214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:27.880214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:01:27.880214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 608/1203] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:27.884751 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23710667-eb4f-4b01-ae58-f0856b816811 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:28.030837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23710667-eb4f-4b01-ae58-f0856b816811 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:28.064708 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:28.071336 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a51f00-07e0-4a1b-b0c4-a6e1d12a31ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:28.072368 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a51f00-07e0-4a1b-b0c4-a6e1d12a31ac None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:28.072842 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a51f00-07e0-4a1b-b0c4-a6e1d12a31ac None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:28.073650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a51f00-07e0-4a1b-b0c4-a6e1d12a31ac None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:28.074219 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a51f00-07e0-4a1b-b0c4-a6e1d12a31ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:28.074618 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a51f00-07e0-4a1b-b0c4-a6e1d12a31ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:28.076050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a51f00-07e0-4a1b-b0c4-a6e1d12a31ac None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:28.102448 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 596/1204] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 15:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:28.110201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533c9c9-d0e9-43be-a89b-433d87decd7e None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:28.199953 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:28.291237 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533c9c9-d0e9-43be-a89b-433d87decd7e None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:28.349814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69442ac0-c6ef-441e-8722-7ea59dfba8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:28.350741 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69442ac0-c6ef-441e-8722-7ea59dfba8c0 None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:28.351091 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69442ac0-c6ef-441e-8722-7ea59dfba8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:28.351368 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69442ac0-c6ef-441e-8722-7ea59dfba8c0 None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:28.351996 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69442ac0-c6ef-441e-8722-7ea59dfba8c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:28.353460 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69442ac0-c6ef-441e-8722-7ea59dfba8c0 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:28.381946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 609/1205] 10.222.0.93 () {60 vars in 1132 bytes} [Mon Aug 1 15:01:28 2022] POST /identity/v3/users => generated 251 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:01:30.765295 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a3213-bc97-4bef-b7f5-51ba34c56395 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:30.765295 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a3213-bc97-4bef-b7f5-51ba34c56395 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:30.765295 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a3213-bc97-4bef-b7f5-51ba34c56395 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:30.766238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 597/1206] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:01:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:01:30.774166 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8435445-d30b-44d2-83a7-e2eb508aad76 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:30.774762 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8435445-d30b-44d2-83a7-e2eb508aad76 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:30.774762 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8435445-d30b-44d2-83a7-e2eb508aad76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:30.795065 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-b8435445-d30b-44d2-83a7-e2eb508aad76 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:30.797194 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b8435445-d30b-44d2-83a7-e2eb508aad76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:30.896649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 610/1207] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:01:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:30.917768 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a3213-bc97-4bef-b7f5-51ba34c56395 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:31.052205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a3213-bc97-4bef-b7f5-51ba34c56395 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:31.095000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0894c-316d-4c89-b5cd-c12f19107366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:31.095867 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0894c-316d-4c89-b5cd-c12f19107366 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:31.096261 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0894c-316d-4c89-b5cd-c12f19107366 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:31.099165 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0894c-316d-4c89-b5cd-c12f19107366 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:31.108501 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0894c-316d-4c89-b5cd-c12f19107366 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:31.114133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-cbd0894c-316d-4c89-b5cd-c12f19107366 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:01:31.114133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:31.114133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:31.114133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:31.114133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:31.114133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:31.114133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:31.114133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:31.114133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:31.114133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:31.114133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:31.114133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:31.114133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:31.114133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:31.114133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:31.114133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:31.115392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:31.115392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:31.115392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:31.115392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:31.115392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:31.115392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:31.115392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:31.115392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:31.115392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:31.115392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:31.115392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:31.115392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:31.115392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:31.115392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:31.115392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:31.115392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:31.116312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:31.116312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:31.116312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:31.116312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:31.116312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:31.116312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:31.116312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:31.116312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:31.116312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:31.116312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:31.116312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:01:31.116312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:01:31.116312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 598/1208] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 15:01:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:31.121696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8435445-d30b-44d2-83a7-e2eb508aad76 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:31.176652 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:31.256957 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8435445-d30b-44d2-83a7-e2eb508aad76 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:31.310405 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00898b7c-7e35-46f7-ad55-3e548a9b8eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:31.311221 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00898b7c-7e35-46f7-ad55-3e548a9b8eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:31.312161 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00898b7c-7e35-46f7-ad55-3e548a9b8eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:31.312625 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00898b7c-7e35-46f7-ad55-3e548a9b8eb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:31.313561 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00898b7c-7e35-46f7-ad55-3e548a9b8eb2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:31.314007 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00898b7c-7e35-46f7-ad55-3e548a9b8eb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:31.316096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00898b7c-7e35-46f7-ad55-3e548a9b8eb2 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:31.336213 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 611/1209] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 15:01:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:31.336483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0894c-316d-4c89-b5cd-c12f19107366 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:31.397266 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:31.458504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0894c-316d-4c89-b5cd-c12f19107366 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:31.506670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6d40a-fa68-4d99-83b6-53ac513b2534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:31.507852 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6d40a-fa68-4d99-83b6-53ac513b2534 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:31.508159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6d40a-fa68-4d99-83b6-53ac513b2534 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:31.508489 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6d40a-fa68-4d99-83b6-53ac513b2534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:31.520178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6d40a-fa68-4d99-83b6-53ac513b2534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:31.521721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6d40a-fa68-4d99-83b6-53ac513b2534 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:31.540590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-32a6d40a-fa68-4d99-83b6-53ac513b2534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:31.540590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:31.540590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:31.540590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:31.540590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:31.540590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:31.540590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:31.540590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:31.540590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:31.540590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:31.540590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:31.540590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:31.540590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:31.540590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:31.540590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:31.541855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:31.541855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:31.541855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:31.541855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:31.541855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:31.541855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:31.541855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:31.541855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:31.541855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:31.541855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:31.541855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:31.541855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:31.541855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:31.541855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:31.541855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:31.541855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:31.541855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:31.542990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:31.542990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:31.542990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:31.542990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:01:31.548502 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 599/1210] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:31.548716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00898b7c-7e35-46f7-ad55-3e548a9b8eb2 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:31.669371 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:31.672892 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00898b7c-7e35-46f7-ad55-3e548a9b8eb2 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:31.713555 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1f515-7e78-4f2a-b396-5c53a89a5966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:31.714174 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1f515-7e78-4f2a-b396-5c53a89a5966 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:31.714456 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1f515-7e78-4f2a-b396-5c53a89a5966 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:31.714723 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1f515-7e78-4f2a-b396-5c53a89a5966 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:31.715423 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df1f515-7e78-4f2a-b396-5c53a89a5966 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:31.715718 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df1f515-7e78-4f2a-b396-5c53a89a5966 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:31.717152 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df1f515-7e78-4f2a-b396-5c53a89a5966 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:31.736522 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 612/1211] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 15:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:31.738721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6d40a-fa68-4d99-83b6-53ac513b2534 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:31.869598 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6d40a-fa68-4d99-83b6-53ac513b2534 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:31.871608 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:31.913911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a14e1-4db1-47ad-9c4f-5339c57dd1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:31.914558 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a14e1-4db1-47ad-9c4f-5339c57dd1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:31.917131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a14e1-4db1-47ad-9c4f-5339c57dd1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:31.917131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a14e1-4db1-47ad-9c4f-5339c57dd1e9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:31.925020 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a14e1-4db1-47ad-9c4f-5339c57dd1e9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:31.926237 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a14e1-4db1-47ad-9c4f-5339c57dd1e9 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:31.941509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-a18a14e1-4db1-47ad-9c4f-5339c57dd1e9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:01:31.941509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:31.941509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:31.941509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:31.941509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:31.941509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:31.941509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:31.941509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:31.941509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:31.941509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:31.941509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:31.941509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:31.941509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:31.941509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:31.941509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:31.941509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:31.942834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:31.942834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:31.942834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:31.942834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:31.942834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:31.942834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:31.942834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:31.942834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:31.942834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:31.942834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:31.942834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:31.942834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:31.942834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:31.942834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:31.942834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:31.942834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:31.942834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:31.942834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:31.944006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:31.944006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:31.944006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:31.944006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:31.944006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:31.944006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:31.944006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:31.944006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:31.944006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:01:31.944006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:01:31.948523 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 600/1212] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:31.948977 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1f515-7e78-4f2a-b396-5c53a89a5966 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:32.082127 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1f515-7e78-4f2a-b396-5c53a89a5966 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:32.127640 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3dcb80-573a-4a70-8160-dec9953914f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:32.128294 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dcb80-573a-4a70-8160-dec9953914f2 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:32.128628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dcb80-573a-4a70-8160-dec9953914f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:32.128930 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dcb80-573a-4a70-8160-dec9953914f2 None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:32.129773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3dcb80-573a-4a70-8160-dec9953914f2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:32.130138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3dcb80-573a-4a70-8160-dec9953914f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:32.132110 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3dcb80-573a-4a70-8160-dec9953914f2 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:32.159993 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 613/1213] 10.222.0.93 () {56 vars in 1143 bytes} [Mon Aug 1 15:01:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:32.166116 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a14e1-4db1-47ad-9c4f-5339c57dd1e9 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:32.301263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a14e1-4db1-47ad-9c4f-5339c57dd1e9 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:32.346544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41e5cf-d93d-471e-b0b5-9ffb70d1ea3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:32.347264 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41e5cf-d93d-471e-b0b5-9ffb70d1ea3d None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:32.348814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41e5cf-d93d-471e-b0b5-9ffb70d1ea3d None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:32.348814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41e5cf-d93d-471e-b0b5-9ffb70d1ea3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:32.390331 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:32.439765 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41e5cf-d93d-471e-b0b5-9ffb70d1ea3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:32.441298 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41e5cf-d93d-471e-b0b5-9ffb70d1ea3d None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:32.605041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 601/1214] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 15:01:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:01:32.616199 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3dcb80-573a-4a70-8160-dec9953914f2 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:32.655461 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:32.793418 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3dcb80-573a-4a70-8160-dec9953914f2 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:32.852295 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cefced-1261-4e0a-952d-3e7a9067bb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:32.853517 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cefced-1261-4e0a-952d-3e7a9067bb33 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:32.854093 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cefced-1261-4e0a-952d-3e7a9067bb33 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:32.855594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cefced-1261-4e0a-952d-3e7a9067bb33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:32.873008 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cefced-1261-4e0a-952d-3e7a9067bb33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:32.874889 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cefced-1261-4e0a-952d-3e7a9067bb33 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:32.901454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-43cefced-1261-4e0a-952d-3e7a9067bb33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:32.901454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:32.901454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:32.901454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:32.901454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:32.901454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:32.901454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:32.901454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:32.901454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:32.901454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:32.901454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:32.901454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:32.901454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:32.901454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:32.901454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:32.904495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:32.904495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:32.904495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:32.904495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:32.904495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:32.904495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:32.904495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:32.904495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:32.904495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:32.904495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:32.904495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:32.904495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:32.904495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:32.904495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:32.904495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:32.904495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:32.904495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:32.906083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:32.906083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:32.906083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:32.906083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:01:32.906083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 614/1215] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:32.913448 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41e5cf-d93d-471e-b0b5-9ffb70d1ea3d None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:32.994031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:33.052332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41e5cf-d93d-471e-b0b5-9ffb70d1ea3d None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:33.114069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364278dd-8b06-4ace-a40e-dd607c885b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:33.115091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364278dd-8b06-4ace-a40e-dd607c885b70 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:33.115608 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364278dd-8b06-4ace-a40e-dd607c885b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:33.116110 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364278dd-8b06-4ace-a40e-dd607c885b70 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:33.117787 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364278dd-8b06-4ace-a40e-dd607c885b70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:33.118362 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364278dd-8b06-4ace-a40e-dd607c885b70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:33.121160 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364278dd-8b06-4ace-a40e-dd607c885b70 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:33.148538 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 602/1216] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 15:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:33.150577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cefced-1261-4e0a-952d-3e7a9067bb33 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:33.270388 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:33.315540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cefced-1261-4e0a-952d-3e7a9067bb33 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:33.358146 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5838df-2a3a-4fb3-ab9c-7bd6256d3310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:33.359883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5838df-2a3a-4fb3-ab9c-7bd6256d3310 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:33.360533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5838df-2a3a-4fb3-ab9c-7bd6256d3310 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:33.361041 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5838df-2a3a-4fb3-ab9c-7bd6256d3310 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:33.373668 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5838df-2a3a-4fb3-ab9c-7bd6256d3310 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:33.376389 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5838df-2a3a-4fb3-ab9c-7bd6256d3310 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:33.397873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-4f5838df-2a3a-4fb3-ab9c-7bd6256d3310 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:33.397873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:33.397873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:33.397873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:33.397873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:33.397873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:33.397873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:33.397873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:33.397873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:33.397873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:33.397873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:33.397873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:33.397873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:33.397873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:33.397873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:33.399448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:33.399448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:33.399448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:33.399448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:33.399448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:33.399448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:33.399448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:33.399448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:33.399448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:33.399448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:33.399448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:33.399448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:33.399448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:33.399448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:33.399448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:33.399448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:33.399448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:33.400662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:33.400662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:33.400662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:33.400662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:33.400662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:33.400662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:33.400662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:33.400662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:01:33.407646 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 615/1217] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 15:01:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:33.408967 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364278dd-8b06-4ace-a40e-dd607c885b70 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:33.563114 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364278dd-8b06-4ace-a40e-dd607c885b70 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:33.594093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:33.613271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff13015-26be-405e-bbc7-b3884795d4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:33.613909 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff13015-26be-405e-bbc7-b3884795d4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:33.614198 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff13015-26be-405e-bbc7-b3884795d4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:33.614470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff13015-26be-405e-bbc7-b3884795d4f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:33.615230 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff13015-26be-405e-bbc7-b3884795d4f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:33.615540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff13015-26be-405e-bbc7-b3884795d4f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:33.617041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff13015-26be-405e-bbc7-b3884795d4f3 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:33.637356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 603/1218] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 15:01:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:33.644334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5838df-2a3a-4fb3-ab9c-7bd6256d3310 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:33.805203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5838df-2a3a-4fb3-ab9c-7bd6256d3310 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:33.848325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:33.859494 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74f196-6a13-40e6-bc5e-be74353989ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:33.860514 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74f196-6a13-40e6-bc5e-be74353989ef None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:33.860963 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74f196-6a13-40e6-bc5e-be74353989ef None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:33.861427 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74f196-6a13-40e6-bc5e-be74353989ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:33.862476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74f196-6a13-40e6-bc5e-be74353989ef None admin] RBAC: Adding query filter params (role.id=6cff92a2d53f4855b38a9d9f11aa19bc, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=a07e3e25e6a8466fb237dcb836aecda2) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:33.862985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74f196-6a13-40e6-bc5e-be74353989ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:33.864707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74f196-6a13-40e6-bc5e-be74353989ef None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:33.872749 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 616/1219] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 15:01:33 2022] GET /identity/v3/role_assignments?user.id=a07e3e25e6a8466fb237dcb836aecda2&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=6cff92a2d53f4855b38a9d9f11aa19bc => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:36.215257 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dcd31-c26f-4f7a-a3d0-8a78c4271d73 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:36.216084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dcd31-c26f-4f7a-a3d0-8a78c4271d73 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:36.216084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dcd31-c26f-4f7a-a3d0-8a78c4271d73 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:36.217380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 604/1220] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:01:36.223958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c9802-9648-4faf-9f10-27f195bb3a6a None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:36.224781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c9802-9648-4faf-9f10-27f195bb3a6a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:36.224781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c9802-9648-4faf-9f10-27f195bb3a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:36.246646 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-cf0c9802-9648-4faf-9f10-27f195bb3a6a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:36.247885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-cf0c9802-9648-4faf-9f10-27f195bb3a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:36.347093 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 617/1221] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:01:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:36.370279 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105dcd31-c26f-4f7a-a3d0-8a78c4271d73 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:36.507580 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105dcd31-c26f-4f7a-a3d0-8a78c4271d73 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:36.558221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a2ffa-d75c-4e46-b969-4a8c531a069f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:36.558952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a2ffa-d75c-4e46-b969-4a8c531a069f None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:36.559251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a2ffa-d75c-4e46-b969-4a8c531a069f None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:36.559641 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a2ffa-d75c-4e46-b969-4a8c531a069f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:36.574808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a2ffa-d75c-4e46-b969-4a8c531a069f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:36.580604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-8e0a2ffa-d75c-4e46-b969-4a8c531a069f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:01:36.580604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:36.580604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:36.580604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:36.580604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:36.580604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:36.580604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:36.580604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:36.580604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:36.580604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:36.580604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:36.580604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:36.580604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:36.580604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:36.580604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:36.580604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:36.591268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:36.591268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:36.591268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:36.591268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:36.591268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:36.591268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:36.591268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:36.591268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:36.591268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:36.591268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:36.591268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:36.591268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:36.591268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:36.591268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:36.591268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:36.591268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:36.592979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:36.592979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:36.592979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:36.592979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:36.592979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:36.592979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:36.592979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:36.592979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:36.592979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:36.592979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:36.592979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:01:36.592979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:01:36.592979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 605/1222] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 15:01:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:36.596783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c9802-9648-4faf-9f10-27f195bb3a6a None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:36.732774 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c9802-9648-4faf-9f10-27f195bb3a6a None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:36.763041 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:36.778415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03d048-ad39-471c-8f8b-7cd3cc238613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:36.779057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03d048-ad39-471c-8f8b-7cd3cc238613 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:36.779340 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03d048-ad39-471c-8f8b-7cd3cc238613 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:36.779637 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03d048-ad39-471c-8f8b-7cd3cc238613 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:36.780397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03d048-ad39-471c-8f8b-7cd3cc238613 None admin] RBAC: Adding query filter params (name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:36.780699 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03d048-ad39-471c-8f8b-7cd3cc238613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:36.782082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03d048-ad39-471c-8f8b-7cd3cc238613 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:36.791679 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 618/1223] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 15:01:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:36.799924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a2ffa-d75c-4e46-b969-4a8c531a069f None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:36.858846 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:36.952352 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a2ffa-d75c-4e46-b969-4a8c531a069f None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:37.000719 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cefb88-9caf-4b78-a66d-3c502555ccd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:37.001490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cefb88-9caf-4b78-a66d-3c502555ccd8 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:37.001682 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cefb88-9caf-4b78-a66d-3c502555ccd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:37.001983 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cefb88-9caf-4b78-a66d-3c502555ccd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:37.017973 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cefb88-9caf-4b78-a66d-3c502555ccd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:37.019507 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cefb88-9caf-4b78-a66d-3c502555ccd8 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:37.047585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-04cefb88-9caf-4b78-a66d-3c502555ccd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:37.047585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:37.047585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:37.047585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:37.047585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:37.047585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:37.047585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:37.047585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:37.047585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:37.047585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:37.047585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:37.047585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:37.047585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:37.047585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:37.047585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:37.048718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:37.048718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:37.048718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:37.048718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:37.048718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:37.048718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:37.048718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:37.048718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:37.048718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:37.048718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:37.048718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:37.048718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:37.048718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:37.048718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:37.048718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:37.048718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:37.048718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:37.050446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:37.050446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:37.050446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:37.050446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:01:37.050446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 606/1224] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:37.056049 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03d048-ad39-471c-8f8b-7cd3cc238613 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:37.199762 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03d048-ad39-471c-8f8b-7cd3cc238613 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:37.243243 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22339097-c3a5-4ff6-a8ef-60fe1bebfeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:37.244087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22339097-c3a5-4ff6-a8ef-60fe1bebfeb2 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:37.244340 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22339097-c3a5-4ff6-a8ef-60fe1bebfeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:37.244653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22339097-c3a5-4ff6-a8ef-60fe1bebfeb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:37.245519 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22339097-c3a5-4ff6-a8ef-60fe1bebfeb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:37.245860 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22339097-c3a5-4ff6-a8ef-60fe1bebfeb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:37.247297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22339097-c3a5-4ff6-a8ef-60fe1bebfeb2 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:37.267836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 619/1225] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 15:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:37.274713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cefb88-9caf-4b78-a66d-3c502555ccd8 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:37.427547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:37.435630 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cefb88-9caf-4b78-a66d-3c502555ccd8 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:37.479007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134e5e9-a7d2-44a8-9a90-35d28e6ae3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:37.479648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134e5e9-a7d2-44a8-9a90-35d28e6ae3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:37.479930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134e5e9-a7d2-44a8-9a90-35d28e6ae3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:37.480202 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134e5e9-a7d2-44a8-9a90-35d28e6ae3b0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:37.488851 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134e5e9-a7d2-44a8-9a90-35d28e6ae3b0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:37.493172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134e5e9-a7d2-44a8-9a90-35d28e6ae3b0 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:37.504419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-5134e5e9-a7d2-44a8-9a90-35d28e6ae3b0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:01:37.504419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:37.504419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:37.504419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:37.504419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:37.504419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:37.504419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:37.504419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:37.504419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:37.504419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:37.504419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:37.504419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:37.504419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:37.504419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:37.504419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:37.504419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:37.505675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:37.505675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:37.505675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:37.505675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:37.505675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:37.505675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:37.505675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:37.505675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:37.505675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:37.505675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:37.505675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:37.505675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:37.505675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:37.505675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:37.505675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:37.505675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:37.505675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:37.505675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:37.506870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:37.506870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:37.506870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:37.506870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:37.506870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:37.506870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:37.506870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:37.506870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:37.506870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:01:37.506870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:01:37.507896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 607/1226] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:37.514712 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22339097-c3a5-4ff6-a8ef-60fe1bebfeb2 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:37.572618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:37.636536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22339097-c3a5-4ff6-a8ef-60fe1bebfeb2 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:37.678085 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd28bc7-88c7-4af5-8f6d-7a13c8b0d4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:37.678736 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd28bc7-88c7-4af5-8f6d-7a13c8b0d4da None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:37.678994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd28bc7-88c7-4af5-8f6d-7a13c8b0d4da None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:37.679279 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd28bc7-88c7-4af5-8f6d-7a13c8b0d4da None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:37.680085 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd28bc7-88c7-4af5-8f6d-7a13c8b0d4da None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:37.680395 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd28bc7-88c7-4af5-8f6d-7a13c8b0d4da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:37.681908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd28bc7-88c7-4af5-8f6d-7a13c8b0d4da None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:37.706040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 620/1227] 10.222.0.93 () {56 vars in 1143 bytes} [Mon Aug 1 15:01:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:37.712082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134e5e9-a7d2-44a8-9a90-35d28e6ae3b0 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:37.842197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134e5e9-a7d2-44a8-9a90-35d28e6ae3b0 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:37.890787 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48adbd70-f11d-404a-8284-3a852e655b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:37.891498 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adbd70-f11d-404a-8284-3a852e655b5b None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:37.891782 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adbd70-f11d-404a-8284-3a852e655b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:37.892064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adbd70-f11d-404a-8284-3a852e655b5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:37.904740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48adbd70-f11d-404a-8284-3a852e655b5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:37.906745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48adbd70-f11d-404a-8284-3a852e655b5b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:37.924713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-48adbd70-f11d-404a-8284-3a852e655b5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:37.924713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:37.924713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:37.924713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:37.924713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:37.924713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:37.924713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:37.924713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:37.924713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:37.924713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:37.924713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:37.924713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:37.924713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:37.924713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:37.924713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:37.926739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:37.926739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:37.926739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:37.926739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:37.926739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:37.926739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:37.926739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:37.926739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:37.926739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:37.926739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:37.926739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:37.926739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:37.926739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:37.926739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:37.926739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:37.926739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:37.926739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:37.928962 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:37.928962 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:37.928962 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:37.928962 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:01:37.928962 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 608/1228] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:37.935473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd28bc7-88c7-4af5-8f6d-7a13c8b0d4da None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:38.075536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd28bc7-88c7-4af5-8f6d-7a13c8b0d4da None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:38.078598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:38.126430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e1b7b-b778-4778-a114-2e8e539618cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:38.127257 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e1b7b-b778-4778-a114-2e8e539618cb None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:38.127691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e1b7b-b778-4778-a114-2e8e539618cb None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:38.128108 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e1b7b-b778-4778-a114-2e8e539618cb None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:38.129237 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e1b7b-b778-4778-a114-2e8e539618cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:38.129677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e1b7b-b778-4778-a114-2e8e539618cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:38.131435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e1b7b-b778-4778-a114-2e8e539618cb None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:38.148697 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 621/1229] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 15:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:38.155834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48adbd70-f11d-404a-8284-3a852e655b5b None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:38.167655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:38.313689 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48adbd70-f11d-404a-8284-3a852e655b5b None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:38.360010 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b67b5a-2a13-44f7-9c13-e2d5728946a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:38.360689 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b67b5a-2a13-44f7-9c13-e2d5728946a1 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:38.360966 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b67b5a-2a13-44f7-9c13-e2d5728946a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:38.361265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b67b5a-2a13-44f7-9c13-e2d5728946a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:38.369789 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b67b5a-2a13-44f7-9c13-e2d5728946a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:38.371598 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b67b5a-2a13-44f7-9c13-e2d5728946a1 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:38.385281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-38b67b5a-2a13-44f7-9c13-e2d5728946a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:38.385281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:38.385281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:38.385281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:38.385281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:38.385281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:38.385281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:38.385281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:38.385281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:38.385281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:38.385281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:38.385281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:38.385281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:38.385281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:38.385281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:38.392062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:38.392062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:38.392062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:38.392062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:38.392062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:38.392062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:38.392062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:38.392062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:38.392062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:38.392062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:38.392062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:38.392062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:38.392062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:38.392062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:38.392062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:38.392062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:38.392062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:38.393186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:38.393186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:38.393186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:38.393186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:38.393186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:38.393186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:38.393186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:38.393186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:01:38.393186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 609/1230] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 15:01:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:38.394186 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e1b7b-b778-4778-a114-2e8e539618cb None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:38.532430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e1b7b-b778-4778-a114-2e8e539618cb None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:38.584622 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015af5b-134e-4ce9-bcb7-19f35ad7d4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:38.585256 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015af5b-134e-4ce9-bcb7-19f35ad7d4fa None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:38.589385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015af5b-134e-4ce9-bcb7-19f35ad7d4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:38.589385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015af5b-134e-4ce9-bcb7-19f35ad7d4fa None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:38.589385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015af5b-134e-4ce9-bcb7-19f35ad7d4fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:38.589385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015af5b-134e-4ce9-bcb7-19f35ad7d4fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:38.591269 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015af5b-134e-4ce9-bcb7-19f35ad7d4fa None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:38.607512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 622/1231] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 15:01:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:38.616880 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b67b5a-2a13-44f7-9c13-e2d5728946a1 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:38.704431 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:38.749658 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b67b5a-2a13-44f7-9c13-e2d5728946a1 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:38.797330 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752f832-1dd9-466d-9251-776e2dd2c80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:38.798213 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752f832-1dd9-466d-9251-776e2dd2c80b None admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:38.798508 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752f832-1dd9-466d-9251-776e2dd2c80b None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:38.798785 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752f832-1dd9-466d-9251-776e2dd2c80b None admin] PATH_INFO: `/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/a07e3e25e6a8466fb237dcb836aecda2/roles/6cff92a2d53f4855b38a9d9f11aa19bc` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:38.819610 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752f832-1dd9-466d-9251-776e2dd2c80b None admin] RBAC: Authorizing `identity:create_grant(project_id=ac9f7c74597642b0acecc956429ac4d8, user_id=a07e3e25e6a8466fb237dcb836aecda2, role_id=6cff92a2d53f4855b38a9d9f11aa19bc)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:38.821274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752f832-1dd9-466d-9251-776e2dd2c80b None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:38.838568 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 610/1232] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 15:01:38 2022] PUT /identity/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/a07e3e25e6a8466fb237dcb836aecda2/roles/6cff92a2d53f4855b38a9d9f11aa19bc => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:01:41.104346 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9db1e5-71c6-47c3-bb8c-9eec6b69520d None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:41.105013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9db1e5-71c6-47c3-bb8c-9eec6b69520d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:41.105013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9db1e5-71c6-47c3-bb8c-9eec6b69520d None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:41.106686 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 623/1233] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:01:41.112799 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f6558-81e6-41ee-8bcd-7a9b20a47b4f None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:41.113327 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f6558-81e6-41ee-8bcd-7a9b20a47b4f None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:41.113327 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f6558-81e6-41ee-8bcd-7a9b20a47b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:41.134977 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-874f6558-81e6-41ee-8bcd-7a9b20a47b4f None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:41.136266 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-874f6558-81e6-41ee-8bcd-7a9b20a47b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:41.221632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 611/1234] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:01:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:41.245450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9db1e5-71c6-47c3-bb8c-9eec6b69520d None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:41.288524 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:41.404215 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9db1e5-71c6-47c3-bb8c-9eec6b69520d None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:41.455221 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69483979-0d4b-418e-a321-c17afb660e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:41.456264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69483979-0d4b-418e-a321-c17afb660e4c None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:41.456635 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69483979-0d4b-418e-a321-c17afb660e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:41.458038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69483979-0d4b-418e-a321-c17afb660e4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:41.471684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69483979-0d4b-418e-a321-c17afb660e4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:41.476873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-69483979-0d4b-418e-a321-c17afb660e4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:01:41.476873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:41.476873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:41.476873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:41.476873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:41.476873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:41.476873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:41.476873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:41.476873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:41.476873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:41.476873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:41.476873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:41.476873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:41.476873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:41.476873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:41.476873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:41.478794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:41.478794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:41.478794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:41.478794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:41.478794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:41.478794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:41.478794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:41.478794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:41.478794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:41.478794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:41.478794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:41.478794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:41.478794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:41.478794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:41.478794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:41.478794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:41.479781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:41.479781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:41.479781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:41.479781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:41.479781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:41.479781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:41.479781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:41.479781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:41.479781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:41.479781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:41.479781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:01:41.479781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:01:41.479781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 624/1235] 10.222.0.93 () {56 vars in 1102 bytes} [Mon Aug 1 15:01:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:41.485257 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f6558-81e6-41ee-8bcd-7a9b20a47b4f None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:41.565823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:41.642865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f6558-81e6-41ee-8bcd-7a9b20a47b4f None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:41.692401 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60e399-c12b-4452-8466-488dcb12cd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:41.693344 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60e399-c12b-4452-8466-488dcb12cd4a None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:41.693815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60e399-c12b-4452-8466-488dcb12cd4a None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:41.694220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60e399-c12b-4452-8466-488dcb12cd4a None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:41.695460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60e399-c12b-4452-8466-488dcb12cd4a None admin] RBAC: Adding query filter params (name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:41.696055 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60e399-c12b-4452-8466-488dcb12cd4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:41.698094 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60e399-c12b-4452-8466-488dcb12cd4a None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:41.712366 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 612/1236] 10.222.0.93 () {56 vars in 1103 bytes} [Mon Aug 1 15:01:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:41.718835 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69483979-0d4b-418e-a321-c17afb660e4c None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:41.822255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:41.840832 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69483979-0d4b-418e-a321-c17afb660e4c None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:41.892089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a2fc7-1a3b-48fd-a14d-d7f36d4a7034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:41.892885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a2fc7-1a3b-48fd-a14d-d7f36d4a7034 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:41.893160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a2fc7-1a3b-48fd-a14d-d7f36d4a7034 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:41.893430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a2fc7-1a3b-48fd-a14d-d7f36d4a7034 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:41.904345 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a2fc7-1a3b-48fd-a14d-d7f36d4a7034 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:41.905863 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a2fc7-1a3b-48fd-a14d-d7f36d4a7034 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:41.928950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-fc2a2fc7-1a3b-48fd-a14d-d7f36d4a7034 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:41.928950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:41.928950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:41.928950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:41.928950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:41.928950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:41.928950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:41.928950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:41.928950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:41.928950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:41.928950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:41.928950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:41.928950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:41.928950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:41.928950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:41.930206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:41.930206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:41.930206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:41.930206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:41.930206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:41.930206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:41.930206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:41.930206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:41.930206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:41.930206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:41.930206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:41.930206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:41.930206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:41.930206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:41.930206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:41.930206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:41.930206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:41.931212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:41.931212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:41.931212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:41.931212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:01:41.931212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 625/1237] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:41.937200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60e399-c12b-4452-8466-488dcb12cd4a None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:42.112492 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60e399-c12b-4452-8466-488dcb12cd4a None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:42.124568 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:42.158636 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf3725-f5ac-4636-b0fb-8951f6cc8d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:42.159460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf3725-f5ac-4636-b0fb-8951f6cc8d72 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:42.159748 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf3725-f5ac-4636-b0fb-8951f6cc8d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:42.160054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf3725-f5ac-4636-b0fb-8951f6cc8d72 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:42.160902 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf3725-f5ac-4636-b0fb-8951f6cc8d72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:42.161258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf3725-f5ac-4636-b0fb-8951f6cc8d72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:42.162766 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf3725-f5ac-4636-b0fb-8951f6cc8d72 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:42.178723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 613/1238] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 15:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:42.187786 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a2fc7-1a3b-48fd-a14d-d7f36d4a7034 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:42.321278 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a2fc7-1a3b-48fd-a14d-d7f36d4a7034 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:42.361958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c525e-62d8-4a26-809f-66d242b31067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:42.362874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c525e-62d8-4a26-809f-66d242b31067 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:42.363200 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c525e-62d8-4a26-809f-66d242b31067 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:42.363494 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c525e-62d8-4a26-809f-66d242b31067 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:42.367850 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:42.380063 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c525e-62d8-4a26-809f-66d242b31067 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:42.382565 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c525e-62d8-4a26-809f-66d242b31067 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:42.399383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-a23c525e-62d8-4a26-809f-66d242b31067 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:01:42.399383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:42.399383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:42.399383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:42.399383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:42.399383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:42.399383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:42.399383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:42.399383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:42.399383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:42.399383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:42.399383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:42.399383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:42.399383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:42.399383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:42.399383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:42.401598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:42.401598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:42.401598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:42.401598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:42.401598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:42.401598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:42.401598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:42.401598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:42.401598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:42.401598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:42.401598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:42.401598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:42.401598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:42.401598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:42.401598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:42.401598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:42.401598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:42.401598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:42.402842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:42.402842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:42.402842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:42.402842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:42.402842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:42.402842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:42.402842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:42.402842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:42.402842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:01:42.402842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:01:42.402842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 626/1239] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:42.408135 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf3725-f5ac-4636-b0fb-8951f6cc8d72 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:42.524111 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf3725-f5ac-4636-b0fb-8951f6cc8d72 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:42.562811 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435bce0d-e5a6-4e8f-b419-06264785e08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:42.563427 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bce0d-e5a6-4e8f-b419-06264785e08d None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:42.563704 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bce0d-e5a6-4e8f-b419-06264785e08d None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:42.563973 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bce0d-e5a6-4e8f-b419-06264785e08d None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:42.564799 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435bce0d-e5a6-4e8f-b419-06264785e08d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:42.565398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435bce0d-e5a6-4e8f-b419-06264785e08d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:42.566631 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435bce0d-e5a6-4e8f-b419-06264785e08d None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:42.586286 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 614/1240] 10.222.0.93 () {56 vars in 1143 bytes} [Mon Aug 1 15:01:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:42.593650 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c525e-62d8-4a26-809f-66d242b31067 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:42.771168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c525e-62d8-4a26-809f-66d242b31067 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:42.812446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69dc8d-9f6b-4c21-bf71-20f3dc8730c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:42.813475 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69dc8d-9f6b-4c21-bf71-20f3dc8730c0 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:42.813956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69dc8d-9f6b-4c21-bf71-20f3dc8730c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:42.814421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69dc8d-9f6b-4c21-bf71-20f3dc8730c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:42.896896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:42.906465 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69dc8d-9f6b-4c21-bf71-20f3dc8730c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:42.907941 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69dc8d-9f6b-4c21-bf71-20f3dc8730c0 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:43.042474 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 627/1241] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 15:01:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:01:43.043846 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435bce0d-e5a6-4e8f-b419-06264785e08d None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:43.079247 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:43.168030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435bce0d-e5a6-4e8f-b419-06264785e08d None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:43.211072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46e626-8818-482b-90c0-25a9950f46af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:43.211911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46e626-8818-482b-90c0-25a9950f46af None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:43.212243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46e626-8818-482b-90c0-25a9950f46af None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:43.212577 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46e626-8818-482b-90c0-25a9950f46af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:43.224208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46e626-8818-482b-90c0-25a9950f46af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:43.225980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46e626-8818-482b-90c0-25a9950f46af None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:43.245916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-9f46e626-8818-482b-90c0-25a9950f46af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:43.245916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:43.245916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:43.245916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:43.245916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:43.245916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:43.245916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:43.245916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:43.245916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:43.245916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:43.245916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:43.245916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:43.245916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:43.245916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:43.245916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:43.247218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:43.247218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:43.247218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:43.247218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:43.247218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:43.247218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:43.247218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:43.247218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:43.247218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:43.247218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:43.247218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:43.247218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:43.247218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:43.247218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:43.247218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:43.247218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:43.247218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:43.248309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:43.248309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:43.248309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:43.248309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:01:43.248309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 615/1242] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:43.254215 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69dc8d-9f6b-4c21-bf71-20f3dc8730c0 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:43.363140 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:43.371914 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69dc8d-9f6b-4c21-bf71-20f3dc8730c0 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:43.417644 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfe3e4-d8fc-4f83-960a-cacce39da600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:43.418267 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfe3e4-d8fc-4f83-960a-cacce39da600 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:43.418554 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfe3e4-d8fc-4f83-960a-cacce39da600 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:43.418813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfe3e4-d8fc-4f83-960a-cacce39da600 None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:43.419518 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfe3e4-d8fc-4f83-960a-cacce39da600 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:43.419811 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfe3e4-d8fc-4f83-960a-cacce39da600 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:43.423889 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfe3e4-d8fc-4f83-960a-cacce39da600 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:43.438028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 628/1243] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 15:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:43.444807 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46e626-8818-482b-90c0-25a9950f46af None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:43.537784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:43.566910 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46e626-8818-482b-90c0-25a9950f46af None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:43.607065 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410d197-3931-48af-b88f-abac85f9d79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:43.607701 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410d197-3931-48af-b88f-abac85f9d79e None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:43.608024 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410d197-3931-48af-b88f-abac85f9d79e None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:43.608324 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410d197-3931-48af-b88f-abac85f9d79e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:43.616642 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410d197-3931-48af-b88f-abac85f9d79e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:43.618328 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410d197-3931-48af-b88f-abac85f9d79e None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:43.631544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-a410d197-3931-48af-b88f-abac85f9d79e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:43.631544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:43.631544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:43.631544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:43.631544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:43.631544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:43.631544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:43.631544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:43.631544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:43.631544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:43.631544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:43.631544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:43.631544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:43.631544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:43.631544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:43.633767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:43.633767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:43.633767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:43.633767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:43.633767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:43.633767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:43.633767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:43.633767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:43.633767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:43.633767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:43.633767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:43.633767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:43.633767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:43.633767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:43.633767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:43.633767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:43.633767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:43.635723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:43.635723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:43.635723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:43.635723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:43.635723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:43.635723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:43.635723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:43.635723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:01:43.638013 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 616/1244] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 15:01:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:43.640783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfe3e4-d8fc-4f83-960a-cacce39da600 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:43.796970 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfe3e4-d8fc-4f83-960a-cacce39da600 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:43.852160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c4d1d-dacf-4dda-a5aa-71fafcdf790d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:43.853028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c4d1d-dacf-4dda-a5aa-71fafcdf790d None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:43.853380 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c4d1d-dacf-4dda-a5aa-71fafcdf790d None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:43.853783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c4d1d-dacf-4dda-a5aa-71fafcdf790d None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:43.854901 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c4d1d-dacf-4dda-a5aa-71fafcdf790d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:43.855276 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c4d1d-dacf-4dda-a5aa-71fafcdf790d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:43.857245 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c4d1d-dacf-4dda-a5aa-71fafcdf790d None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:43.877621 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 629/1245] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 15:01:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:43.884243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410d197-3931-48af-b88f-abac85f9d79e None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:44.019681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410d197-3931-48af-b88f-abac85f9d79e None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:44.065559 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21543163-8cac-4612-a65d-b60f7f41d444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:44.066722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21543163-8cac-4612-a65d-b60f7f41d444 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:44.067287 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21543163-8cac-4612-a65d-b60f7f41d444 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:44.068142 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21543163-8cac-4612-a65d-b60f7f41d444 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:44.070066 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21543163-8cac-4612-a65d-b60f7f41d444 None admin] RBAC: Adding query filter params (role.id=6cff92a2d53f4855b38a9d9f11aa19bc, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=a07e3e25e6a8466fb237dcb836aecda2) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:44.070747 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21543163-8cac-4612-a65d-b60f7f41d444 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:44.074365 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21543163-8cac-4612-a65d-b60f7f41d444 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:44.087930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 617/1246] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 15:01:43 2022] GET /identity/v3/role_assignments?user.id=a07e3e25e6a8466fb237dcb836aecda2&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=6cff92a2d53f4855b38a9d9f11aa19bc => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:46.569751 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100be8e9-88b3-4e88-b490-89899254f718 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:46.570376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100be8e9-88b3-4e88-b490-89899254f718 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:46.570376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100be8e9-88b3-4e88-b490-89899254f718 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:46.571767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 630/1247] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:01:46.579117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b2032-0845-4860-b04f-a3b10ca17075 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:46.579701 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b2032-0845-4860-b04f-a3b10ca17075 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:46.579701 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b2032-0845-4860-b04f-a3b10ca17075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:46.587325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:46.605475 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-2f1b2032-0845-4860-b04f-a3b10ca17075 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:46.607125 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-2f1b2032-0845-4860-b04f-a3b10ca17075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:46.711704 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 618/1248] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:01:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:46.737045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100be8e9-88b3-4e88-b490-89899254f718 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:46.766840 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:46.886516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100be8e9-88b3-4e88-b490-89899254f718 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:46.938332 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86ea67-b044-4173-bbe1-896f14d04c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:46.938971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86ea67-b044-4173-bbe1-896f14d04c7d None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:46.939235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86ea67-b044-4173-bbe1-896f14d04c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:46.939712 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86ea67-b044-4173-bbe1-896f14d04c7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:46.951575 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86ea67-b044-4173-bbe1-896f14d04c7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:46.956955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-bf86ea67-b044-4173-bbe1-896f14d04c7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:46.956955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:46.956955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:46.956955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:46.956955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:46.956955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:46.956955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:46.956955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:46.956955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:46.956955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:46.956955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:46.956955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:46.956955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:46.956955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:46.956955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:46.956955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:46.958759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:46.958759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:46.958759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:46.958759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:46.958759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:46.958759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:46.958759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:46.958759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:46.958759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:46.958759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:46.958759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:46.958759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:46.958759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:46.958759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:46.958759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:46.958759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:46.959995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:46.959995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:46.959995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:46.959995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:46.959995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:46.959995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:46.959995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:46.959995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:46.959995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:46.959995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:46.959995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:46.959995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:01:46.959995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 631/1249] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 15:01:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:46.965022 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b2032-0845-4860-b04f-a3b10ca17075 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:47.123943 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b2032-0845-4860-b04f-a3b10ca17075 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:47.172096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:47.179249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c63114-4f5a-4f1f-b085-135ff0a50b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:47.179878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c63114-4f5a-4f1f-b085-135ff0a50b0e None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:47.180208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c63114-4f5a-4f1f-b085-135ff0a50b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:47.180452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c63114-4f5a-4f1f-b085-135ff0a50b0e None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:47.181220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c63114-4f5a-4f1f-b085-135ff0a50b0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:47.181525 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c63114-4f5a-4f1f-b085-135ff0a50b0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:47.183062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c63114-4f5a-4f1f-b085-135ff0a50b0e None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:47.194321 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 619/1250] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 15:01:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:47.200595 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86ea67-b044-4173-bbe1-896f14d04c7d None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:47.327473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:47.378441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86ea67-b044-4173-bbe1-896f14d04c7d None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:47.428368 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495537a-8071-4efe-8d2e-7f6deec9582e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:47.429099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495537a-8071-4efe-8d2e-7f6deec9582e None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:47.429379 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495537a-8071-4efe-8d2e-7f6deec9582e None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:47.429658 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495537a-8071-4efe-8d2e-7f6deec9582e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:47.441623 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0495537a-8071-4efe-8d2e-7f6deec9582e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:47.443170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0495537a-8071-4efe-8d2e-7f6deec9582e None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:47.465071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-0495537a-8071-4efe-8d2e-7f6deec9582e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:47.465071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:47.465071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:47.465071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:47.465071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:47.465071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:47.465071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:47.465071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:47.465071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:47.465071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:47.465071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:47.465071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:47.465071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:47.465071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:47.465071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:47.469077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:47.469077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:47.469077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:47.469077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:47.469077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:47.469077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:47.469077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:47.469077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:47.469077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:47.469077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:47.469077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:47.469077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:47.469077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:47.469077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:47.469077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:47.469077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:47.469077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:47.470207 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:47.470207 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:47.470207 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:47.470207 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:01:47.470207 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 632/1251] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:47.473311 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c63114-4f5a-4f1f-b085-135ff0a50b0e None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:47.615487 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c63114-4f5a-4f1f-b085-135ff0a50b0e None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:47.664344 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbaf8d-971d-4f9b-a69d-7db23642bec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:47.665238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbaf8d-971d-4f9b-a69d-7db23642bec8 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:47.665746 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbaf8d-971d-4f9b-a69d-7db23642bec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:47.666252 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbaf8d-971d-4f9b-a69d-7db23642bec8 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:47.667577 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dbaf8d-971d-4f9b-a69d-7db23642bec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:47.668142 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dbaf8d-971d-4f9b-a69d-7db23642bec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:47.670710 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dbaf8d-971d-4f9b-a69d-7db23642bec8 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:47.691965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 620/1252] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 15:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:47.698860 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495537a-8071-4efe-8d2e-7f6deec9582e None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:47.852310 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495537a-8071-4efe-8d2e-7f6deec9582e None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:47.892593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:47.904274 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428452b1-b5be-46a2-bfc8-b76f96028a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:47.905888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428452b1-b5be-46a2-bfc8-b76f96028a16 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:47.906398 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428452b1-b5be-46a2-bfc8-b76f96028a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:47.906877 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428452b1-b5be-46a2-bfc8-b76f96028a16 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:47.920835 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428452b1-b5be-46a2-bfc8-b76f96028a16 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:47.922422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428452b1-b5be-46a2-bfc8-b76f96028a16 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:47.940188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-428452b1-b5be-46a2-bfc8-b76f96028a16 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:01:47.940188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:47.940188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:47.940188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:47.940188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:47.940188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:47.940188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:47.940188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:47.940188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:47.940188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:47.940188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:47.940188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:47.940188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:47.940188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:47.940188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:47.940188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:47.941452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:47.941452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:47.941452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:47.941452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:47.941452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:47.941452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:47.941452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:47.941452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:47.941452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:47.941452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:47.941452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:47.941452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:47.941452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:47.941452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:47.941452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:47.941452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:47.941452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:47.941452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:47.942529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:47.942529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:47.942529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:47.942529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:47.942529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:47.942529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:47.942529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:47.942529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:47.942529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:01:47.942529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:01:47.942529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 633/1253] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:47.947997 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbaf8d-971d-4f9b-a69d-7db23642bec8 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:48.019776 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:48.081573 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbaf8d-971d-4f9b-a69d-7db23642bec8 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:48.130282 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95fff1-0df0-4afa-bb6a-2b18ae3944d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:48.130903 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95fff1-0df0-4afa-bb6a-2b18ae3944d2 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:48.131221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95fff1-0df0-4afa-bb6a-2b18ae3944d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:48.131443 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95fff1-0df0-4afa-bb6a-2b18ae3944d2 None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:48.132248 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95fff1-0df0-4afa-bb6a-2b18ae3944d2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:48.132586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95fff1-0df0-4afa-bb6a-2b18ae3944d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:48.134031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95fff1-0df0-4afa-bb6a-2b18ae3944d2 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:48.156409 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 621/1254] 10.222.0.93 () {56 vars in 1143 bytes} [Mon Aug 1 15:01:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:48.163121 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428452b1-b5be-46a2-bfc8-b76f96028a16 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:48.298701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428452b1-b5be-46a2-bfc8-b76f96028a16 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:48.347486 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79fb8f-3ee6-43e9-a356-62624d55f33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:48.348158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79fb8f-3ee6-43e9-a356-62624d55f33f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:48.348526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79fb8f-3ee6-43e9-a356-62624d55f33f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:48.348928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79fb8f-3ee6-43e9-a356-62624d55f33f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:48.444611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:48.463769 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79fb8f-3ee6-43e9-a356-62624d55f33f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:48.465606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79fb8f-3ee6-43e9-a356-62624d55f33f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:48.621453 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 634/1255] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 15:01:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:01:48.632998 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95fff1-0df0-4afa-bb6a-2b18ae3944d2 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:48.834539 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95fff1-0df0-4afa-bb6a-2b18ae3944d2 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:48.852645 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:48.887585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7d8dd-d3f1-458b-8850-ab739333b0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:48.888568 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7d8dd-d3f1-458b-8850-ab739333b0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:48.888568 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7d8dd-d3f1-458b-8850-ab739333b0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:48.890397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7d8dd-d3f1-458b-8850-ab739333b0a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:48.906021 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7d8dd-d3f1-458b-8850-ab739333b0a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:48.908809 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7d8dd-d3f1-458b-8850-ab739333b0a5 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:48.931486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-30d7d8dd-d3f1-458b-8850-ab739333b0a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:48.931486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:48.931486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:48.931486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:48.931486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:48.931486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:48.931486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:48.931486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:48.931486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:48.931486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:48.931486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:48.931486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:48.931486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:48.931486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:48.931486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:48.932899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:48.932899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:48.932899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:48.932899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:48.932899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:48.932899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:48.932899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:48.932899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:48.932899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:48.932899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:48.932899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:48.932899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:48.932899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:48.932899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:48.932899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:48.932899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:48.932899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:48.933862 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:48.933862 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:48.933862 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:48.933862 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:01:48.933862 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 622/1256] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:48.939299 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79fb8f-3ee6-43e9-a356-62624d55f33f None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:49.091339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:49.113266 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79fb8f-3ee6-43e9-a356-62624d55f33f None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:49.168648 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556edb9-0497-464d-8b19-3046a66c974f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:49.169379 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556edb9-0497-464d-8b19-3046a66c974f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:49.169596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556edb9-0497-464d-8b19-3046a66c974f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:49.170036 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556edb9-0497-464d-8b19-3046a66c974f None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:49.171728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556edb9-0497-464d-8b19-3046a66c974f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:49.172227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556edb9-0497-464d-8b19-3046a66c974f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:49.175547 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556edb9-0497-464d-8b19-3046a66c974f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:49.189887 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 635/1257] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 15:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:49.200099 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7d8dd-d3f1-458b-8850-ab739333b0a5 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:49.378438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7d8dd-d3f1-458b-8850-ab739333b0a5 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:49.453840 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfb575-f772-4bf8-ba1a-db162a3c356f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:49.454478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfb575-f772-4bf8-ba1a-db162a3c356f None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:49.454804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfb575-f772-4bf8-ba1a-db162a3c356f None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:49.455096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfb575-f772-4bf8-ba1a-db162a3c356f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:49.466815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfb575-f772-4bf8-ba1a-db162a3c356f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:49.469173 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfb575-f772-4bf8-ba1a-db162a3c356f None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:49.485735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-a7dfb575-f772-4bf8-ba1a-db162a3c356f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:49.485735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:49.485735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:49.485735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:49.485735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:49.485735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:49.485735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:49.485735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:49.485735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:49.485735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:49.485735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:49.485735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:49.485735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:49.485735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:49.485735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:49.487808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:49.487808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:49.487808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:49.487808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:49.487808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:49.487808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:49.487808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:49.487808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:49.487808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:49.487808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:49.487808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:49.487808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:49.487808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:49.487808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:49.487808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:49.487808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:49.487808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:49.489276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:49.489276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:49.489276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:49.489276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:49.489276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:49.489276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:49.489276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:49.489276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:01:49.489276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 623/1258] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 15:01:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:49.497801 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556edb9-0497-464d-8b19-3046a66c974f None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:49.693169 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556edb9-0497-464d-8b19-3046a66c974f None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:49.757022 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e47ca6-bece-40b6-939c-c1fef20492ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:49.758163 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e47ca6-bece-40b6-939c-c1fef20492ec None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:49.758583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e47ca6-bece-40b6-939c-c1fef20492ec None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:49.759077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e47ca6-bece-40b6-939c-c1fef20492ec None admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:49.760400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e47ca6-bece-40b6-939c-c1fef20492ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:49.760961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e47ca6-bece-40b6-939c-c1fef20492ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:49.763773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e47ca6-bece-40b6-939c-c1fef20492ec None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:49.782764 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 636/1259] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 15:01:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:49.790207 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfb575-f772-4bf8-ba1a-db162a3c356f None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:49.828398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:49.943222 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfb575-f772-4bf8-ba1a-db162a3c356f None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:50.001286 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3687ce9-437c-4d14-b51e-82c0cb70b1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:50.002531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3687ce9-437c-4d14-b51e-82c0cb70b1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:50.003099 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3687ce9-437c-4d14-b51e-82c0cb70b1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:50.003757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3687ce9-437c-4d14-b51e-82c0cb70b1e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:50.005666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3687ce9-437c-4d14-b51e-82c0cb70b1e2 None admin] RBAC: Adding query filter params (role.id=5456887a306649bf80a9ea7bd4282e97, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=a07e3e25e6a8466fb237dcb836aecda2) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:50.006260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3687ce9-437c-4d14-b51e-82c0cb70b1e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:50.009598 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3687ce9-437c-4d14-b51e-82c0cb70b1e2 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:50.018626 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 624/1260] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 15:01:49 2022] GET /identity/v3/role_assignments?user.id=a07e3e25e6a8466fb237dcb836aecda2&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:52.593428 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c00289-34bb-4de4-8f63-5fdd9eba6eca None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:52.594134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c00289-34bb-4de4-8f63-5fdd9eba6eca None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:52.594134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c00289-34bb-4de4-8f63-5fdd9eba6eca None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:52.595394 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 637/1261] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:01:52.601587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d8f3e-9afb-4676-acc0-c6954b0f5d8f None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:52.602162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d8f3e-9afb-4676-acc0-c6954b0f5d8f None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:52.602162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d8f3e-9afb-4676-acc0-c6954b0f5d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:52.624613 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-1d4d8f3e-9afb-4676-acc0-c6954b0f5d8f None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:52.625966 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-1d4d8f3e-9afb-4676-acc0-c6954b0f5d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:52.736214 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 625/1262] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:01:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:52.759950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c00289-34bb-4de4-8f63-5fdd9eba6eca None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:52.776421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:52.886861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c00289-34bb-4de4-8f63-5fdd9eba6eca None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:52.931161 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895d773-2d05-482b-9548-e1e4494b411a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:52.931806 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895d773-2d05-482b-9548-e1e4494b411a None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:52.932084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895d773-2d05-482b-9548-e1e4494b411a None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:52.932357 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895d773-2d05-482b-9548-e1e4494b411a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:52.947078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a895d773-2d05-482b-9548-e1e4494b411a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:52.952031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-a895d773-2d05-482b-9548-e1e4494b411a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:52.952031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:52.952031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:52.952031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:52.952031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:52.952031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:52.952031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:52.952031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:52.952031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:52.952031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:52.952031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:52.952031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:52.952031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:52.952031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:52.952031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:52.952031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:52.953352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:52.953352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:52.953352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:52.953352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:52.953352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:52.953352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:52.953352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:52.953352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:52.953352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:52.953352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:52.953352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:52.953352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:52.953352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:52.953352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:52.953352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:52.953352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:52.954611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:52.954611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:52.954611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:52.954611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:52.954611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:52.954611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:52.954611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:52.954611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:52.954611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:52.954611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:52.954611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:52.954611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:01:52.954611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 638/1263] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 15:01:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:52.959529 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d8f3e-9afb-4676-acc0-c6954b0f5d8f None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:52.981034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:53.093482 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d8f3e-9afb-4676-acc0-c6954b0f5d8f None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:53.129666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3aa174-c619-4d31-903e-61c636ec32a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:53.130319 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3aa174-c619-4d31-903e-61c636ec32a1 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:53.130560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3aa174-c619-4d31-903e-61c636ec32a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:53.130833 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3aa174-c619-4d31-903e-61c636ec32a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:53.131533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3aa174-c619-4d31-903e-61c636ec32a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:53.131831 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3aa174-c619-4d31-903e-61c636ec32a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:53.133212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3aa174-c619-4d31-903e-61c636ec32a1 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:53.144078 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 626/1264] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 15:01:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:53.149961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895d773-2d05-482b-9548-e1e4494b411a None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:53.222993 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:53.258214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895d773-2d05-482b-9548-e1e4494b411a None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:53.294104 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467811e9-12bc-4e5f-b28a-9314c1f1a94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:53.294745 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467811e9-12bc-4e5f-b28a-9314c1f1a94f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:53.295045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467811e9-12bc-4e5f-b28a-9314c1f1a94f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:53.295302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467811e9-12bc-4e5f-b28a-9314c1f1a94f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:53.306691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467811e9-12bc-4e5f-b28a-9314c1f1a94f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:53.308171 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467811e9-12bc-4e5f-b28a-9314c1f1a94f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:53.322942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-467811e9-12bc-4e5f-b28a-9314c1f1a94f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:53.322942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:53.322942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:53.322942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:53.322942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:53.322942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:53.322942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:53.322942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:53.322942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:53.322942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:53.322942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:53.322942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:53.322942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:53.322942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:53.322942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:53.324120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:53.324120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:53.324120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:53.324120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:53.324120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:53.324120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:53.324120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:53.324120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:53.324120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:53.324120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:53.324120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:53.324120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:53.324120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:53.324120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:53.324120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:53.324120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:53.324120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:53.325109 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:53.325109 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:53.325109 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:53.325109 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:01:53.325109 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 639/1265] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:53.334149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3aa174-c619-4d31-903e-61c636ec32a1 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:53.434806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:53.456112 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3aa174-c619-4d31-903e-61c636ec32a1 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:53.495608 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588818b-e330-442f-aa16-d7590555430c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:53.496247 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588818b-e330-442f-aa16-d7590555430c None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:53.496554 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588818b-e330-442f-aa16-d7590555430c None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:53.496833 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588818b-e330-442f-aa16-d7590555430c None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:53.497558 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588818b-e330-442f-aa16-d7590555430c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:53.497863 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588818b-e330-442f-aa16-d7590555430c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:53.499815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588818b-e330-442f-aa16-d7590555430c None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:53.517431 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 627/1266] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 15:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:53.526713 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467811e9-12bc-4e5f-b28a-9314c1f1a94f None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:53.648806 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467811e9-12bc-4e5f-b28a-9314c1f1a94f None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:53.670580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:53.686249 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e3042-8385-4fd0-b4af-ed0e92b827b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:53.686891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e3042-8385-4fd0-b4af-ed0e92b827b5 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:53.687199 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e3042-8385-4fd0-b4af-ed0e92b827b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:53.687463 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e3042-8385-4fd0-b4af-ed0e92b827b5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:53.695793 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e3042-8385-4fd0-b4af-ed0e92b827b5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:53.697272 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e3042-8385-4fd0-b4af-ed0e92b827b5 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:53.710075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-810e3042-8385-4fd0-b4af-ed0e92b827b5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:01:53.710075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:53.710075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:53.710075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:53.710075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:53.710075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:53.710075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:53.710075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:53.710075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:53.710075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:53.710075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:53.710075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:53.710075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:53.710075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:53.710075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:53.710075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:53.711260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:53.711260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:53.711260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:53.711260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:53.711260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:53.711260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:53.711260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:53.711260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:53.711260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:53.711260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:53.711260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:53.711260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:53.711260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:53.711260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:53.711260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:53.711260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:53.711260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:53.711260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:53.712450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:53.712450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:53.712450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:53.712450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:53.712450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:53.712450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:53.712450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:53.712450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:53.712450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:01:53.712450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:01:53.712450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 640/1267] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:53.717331 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588818b-e330-442f-aa16-d7590555430c None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:53.829442 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588818b-e330-442f-aa16-d7590555430c None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:53.864141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93691c99-69f0-4735-961d-6cb3e44163ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:53.864779 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93691c99-69f0-4735-961d-6cb3e44163ea None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:53.865060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93691c99-69f0-4735-961d-6cb3e44163ea None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:53.865335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93691c99-69f0-4735-961d-6cb3e44163ea None admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:53.866133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93691c99-69f0-4735-961d-6cb3e44163ea None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:53.866441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93691c99-69f0-4735-961d-6cb3e44163ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:53.868187 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93691c99-69f0-4735-961d-6cb3e44163ea None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:53.880988 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:53.891834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 628/1268] 10.222.0.93 () {56 vars in 1143 bytes} [Mon Aug 1 15:01:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:53.897517 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e3042-8385-4fd0-b4af-ed0e92b827b5 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:54.005479 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e3042-8385-4fd0-b4af-ed0e92b827b5 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:54.048182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd18bf6-d135-4e2e-b72e-0b99a8919245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:54.048993 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd18bf6-d135-4e2e-b72e-0b99a8919245 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:54.049361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd18bf6-d135-4e2e-b72e-0b99a8919245 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:54.049697 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd18bf6-d135-4e2e-b72e-0b99a8919245 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:54.064819 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd18bf6-d135-4e2e-b72e-0b99a8919245 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:54.066721 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd18bf6-d135-4e2e-b72e-0b99a8919245 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:54.089922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-fcd18bf6-d135-4e2e-b72e-0b99a8919245 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:54.089922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:54.089922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:54.089922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:54.089922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:54.089922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:54.089922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:54.089922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:54.089922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:54.089922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:54.089922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:54.089922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:54.089922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:54.089922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:54.089922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:54.091447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:54.091447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:54.091447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:54.091447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:54.091447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:54.091447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:54.091447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:54.091447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:54.091447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:54.091447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:54.091447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:54.091447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:54.091447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:54.091447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:54.091447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:54.091447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:54.091447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:54.093054 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:54.093054 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:54.093054 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:54.093054 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:01:54.093054 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 641/1269] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:54.099201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93691c99-69f0-4735-961d-6cb3e44163ea None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:54.226782 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93691c99-69f0-4735-961d-6cb3e44163ea None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:54.261956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439695e-d4ae-4b1d-9a73-fe7cb3a041cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:54.262487 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439695e-d4ae-4b1d-9a73-fe7cb3a041cb None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:54.262770 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439695e-d4ae-4b1d-9a73-fe7cb3a041cb None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:54.263038 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439695e-d4ae-4b1d-9a73-fe7cb3a041cb None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:54.263817 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439695e-d4ae-4b1d-9a73-fe7cb3a041cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:54.264674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439695e-d4ae-4b1d-9a73-fe7cb3a041cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:54.266088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439695e-d4ae-4b1d-9a73-fe7cb3a041cb None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:54.280768 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 629/1270] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 15:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:54.287719 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd18bf6-d135-4e2e-b72e-0b99a8919245 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:54.324954 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:54.407089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd18bf6-d135-4e2e-b72e-0b99a8919245 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:54.445385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04f772-c359-4e51-9e0d-c617ad2ee6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:54.446022 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04f772-c359-4e51-9e0d-c617ad2ee6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:54.446302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04f772-c359-4e51-9e0d-c617ad2ee6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:54.446623 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04f772-c359-4e51-9e0d-c617ad2ee6c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:54.454197 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04f772-c359-4e51-9e0d-c617ad2ee6c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:54.455659 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04f772-c359-4e51-9e0d-c617ad2ee6c8 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:54.468050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-bf04f772-c359-4e51-9e0d-c617ad2ee6c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:54.468050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:54.468050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:54.468050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:54.468050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:54.468050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:54.468050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:54.468050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:54.468050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:54.468050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:54.468050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:54.468050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:54.468050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:54.468050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:54.468050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:54.469549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:54.469549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:54.469549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:54.469549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:54.469549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:54.469549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:54.469549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:54.469549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:54.469549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:54.469549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:54.469549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:54.469549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:54.469549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:54.469549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:54.469549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:54.469549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:54.469549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:54.470478 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:54.470478 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:54.470478 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:54.470478 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:54.470478 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:54.470478 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:54.470478 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:54.470478 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:01:54.470478 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 642/1271] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 15:01:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:54.475240 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439695e-d4ae-4b1d-9a73-fe7cb3a041cb None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:54.540205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:54.588188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439695e-d4ae-4b1d-9a73-fe7cb3a041cb None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:54.626329 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b6d2a-fb6c-4e2a-af21-79370e82f842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:54.626945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b6d2a-fb6c-4e2a-af21-79370e82f842 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:54.627216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b6d2a-fb6c-4e2a-af21-79370e82f842 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:54.627484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b6d2a-fb6c-4e2a-af21-79370e82f842 None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:54.628230 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b6d2a-fb6c-4e2a-af21-79370e82f842 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:54.628558 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b6d2a-fb6c-4e2a-af21-79370e82f842 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:54.629988 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b6d2a-fb6c-4e2a-af21-79370e82f842 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:54.643325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 630/1272] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 15:01:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:54.649587 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04f772-c359-4e51-9e0d-c617ad2ee6c8 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:54.752224 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:54.764868 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04f772-c359-4e51-9e0d-c617ad2ee6c8 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:54.805122 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588809b-cdc8-4d20-b579-65a902660c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:54.805735 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588809b-cdc8-4d20-b579-65a902660c8c None admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:54.806011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588809b-cdc8-4d20-b579-65a902660c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:54.806284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588809b-cdc8-4d20-b579-65a902660c8c None admin] PATH_INFO: `/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/a07e3e25e6a8466fb237dcb836aecda2/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:54.808243 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588809b-cdc8-4d20-b579-65a902660c8c None admin] RBAC: Authorizing `identity:create_grant(project_id=ac9f7c74597642b0acecc956429ac4d8, user_id=a07e3e25e6a8466fb237dcb836aecda2, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:54.809733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588809b-cdc8-4d20-b579-65a902660c8c None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:54.824820 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 643/1273] 10.222.0.93 () {58 vars in 1439 bytes} [Mon Aug 1 15:01:54 2022] PUT /identity/v3/projects/ac9f7c74597642b0acecc956429ac4d8/users/a07e3e25e6a8466fb237dcb836aecda2/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:01:56.865724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d9157-2453-45f9-8c0f-884b3d6b331b None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:56.866325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d9157-2453-45f9-8c0f-884b3d6b331b None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:56.866325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d9157-2453-45f9-8c0f-884b3d6b331b None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:56.867899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 631/1274] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:01:56.874814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b156d72-3c70-4bc8-8040-d27e64ac7393 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:56.875386 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b156d72-3c70-4bc8-8040-d27e64ac7393 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:56.875606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b156d72-3c70-4bc8-8040-d27e64ac7393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:56.897541 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-2b156d72-3c70-4bc8-8040-d27e64ac7393 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:56.898737 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-2b156d72-3c70-4bc8-8040-d27e64ac7393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:56.974426 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 644/1275] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:01:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:56.999556 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d9157-2453-45f9-8c0f-884b3d6b331b None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:57.137238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d9157-2453-45f9-8c0f-884b3d6b331b None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:57.160401 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:57.178913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add4590-cab1-42c3-bac2-b8c3ea5b4af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:57.179576 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add4590-cab1-42c3-bac2-b8c3ea5b4af8 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:57.179911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add4590-cab1-42c3-bac2-b8c3ea5b4af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:57.180108 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add4590-cab1-42c3-bac2-b8c3ea5b4af8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:57.192309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add4590-cab1-42c3-bac2-b8c3ea5b4af8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:57.197398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-7add4590-cab1-42c3-bac2-b8c3ea5b4af8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:57.197398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:57.197398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:57.197398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:57.197398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:57.197398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:57.197398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:57.197398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:57.197398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:57.197398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:57.197398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:57.197398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:57.197398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:57.197398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:57.197398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:57.197398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:57.198670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:57.198670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:57.198670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:57.198670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:57.198670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:57.198670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:57.198670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:57.198670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:57.198670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:57.198670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:57.198670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:57.198670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:57.198670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:57.198670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:57.198670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:57.198670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:57.199585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:57.199585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:57.199585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:57.199585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:57.199585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:57.199585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:57.199585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:57.199585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:57.199585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:57.199585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:57.199585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:57.199585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:01:57.199585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 632/1276] 10.222.0.93 () {56 vars in 1096 bytes} [Mon Aug 1 15:01:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:57.204956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b156d72-3c70-4bc8-8040-d27e64ac7393 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:57.292840 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:57.316738 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b156d72-3c70-4bc8-8040-d27e64ac7393 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:57.364029 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf74f68-a1c3-4784-a841-111f4d0ee6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:57.364669 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf74f68-a1c3-4784-a841-111f4d0ee6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:57.364956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf74f68-a1c3-4784-a841-111f4d0ee6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:57.365223 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf74f68-a1c3-4784-a841-111f4d0ee6f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:57.365937 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf74f68-a1c3-4784-a841-111f4d0ee6f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:57.366235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf74f68-a1c3-4784-a841-111f4d0ee6f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:57.367888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf74f68-a1c3-4784-a841-111f4d0ee6f9 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:57.376681 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 645/1277] 10.222.0.93 () {56 vars in 1099 bytes} [Mon Aug 1 15:01:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:57.382884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add4590-cab1-42c3-bac2-b8c3ea5b4af8 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:57.498847 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add4590-cab1-42c3-bac2-b8c3ea5b4af8 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:57.533834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde80eb5-019e-49e8-86f5-b67291436853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:57.534470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde80eb5-019e-49e8-86f5-b67291436853 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:57.534751 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde80eb5-019e-49e8-86f5-b67291436853 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:57.535028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde80eb5-019e-49e8-86f5-b67291436853 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:57.546850 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde80eb5-019e-49e8-86f5-b67291436853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:57.548345 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde80eb5-019e-49e8-86f5-b67291436853 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:57.564400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-cde80eb5-019e-49e8-86f5-b67291436853 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:57.564400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:57.564400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:57.564400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:57.564400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:57.564400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:57.564400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:57.564400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:57.564400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:57.564400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:57.564400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:57.564400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:57.564400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:57.564400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:57.564400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:57.566071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:57.566071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:57.566071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:57.566071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:57.566071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:57.566071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:57.566071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:57.566071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:57.566071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:57.566071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:57.566071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:57.566071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:57.566071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:57.566071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:57.566071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:57.566071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:57.566071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:57.567364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:57.567364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:57.567364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:57.567364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:01:57.567364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 633/1278] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:57.571629 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf74f68-a1c3-4784-a841-111f4d0ee6f9 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:57.676243 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf74f68-a1c3-4784-a841-111f4d0ee6f9 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:57.710485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485a810-796d-4f3a-99fc-b6b20c88ed1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:57.711114 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485a810-796d-4f3a-99fc-b6b20c88ed1b None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:57.711402 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485a810-796d-4f3a-99fc-b6b20c88ed1b None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:57.711685 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485a810-796d-4f3a-99fc-b6b20c88ed1b None admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:57.712384 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485a810-796d-4f3a-99fc-b6b20c88ed1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:57.712732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485a810-796d-4f3a-99fc-b6b20c88ed1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:57.714121 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485a810-796d-4f3a-99fc-b6b20c88ed1b None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:57.729134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 646/1279] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 15:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:57.735432 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde80eb5-019e-49e8-86f5-b67291436853 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:57.767160 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:57.839378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde80eb5-019e-49e8-86f5-b67291436853 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:57.874475 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f4058-d8cf-43fe-9aee-0e7449584b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:57.875224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f4058-d8cf-43fe-9aee-0e7449584b03 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:57.875375 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f4058-d8cf-43fe-9aee-0e7449584b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:57.875650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f4058-d8cf-43fe-9aee-0e7449584b03 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:57.883822 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f4058-d8cf-43fe-9aee-0e7449584b03 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:57.885325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f4058-d8cf-43fe-9aee-0e7449584b03 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:57.897268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-d65f4058-d8cf-43fe-9aee-0e7449584b03 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:01:57.897268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:57.897268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:57.897268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:57.897268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:57.897268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:57.897268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:57.897268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:57.897268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:57.897268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:57.897268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:57.897268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:57.897268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:57.897268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:57.897268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:57.897268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:57.898740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:57.898740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:57.898740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:57.898740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:57.898740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:57.898740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:57.898740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:57.898740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:57.898740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:57.898740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:57.898740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:57.898740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:57.898740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:57.898740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:57.898740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:57.898740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:57.898740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:57.898740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:57.899990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:57.899990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:57.899990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:57.899990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:57.899990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:57.899990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:57.899990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:57.899990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:57.899990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:01:57.899990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:01:57.899990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 634/1280] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:57.905092 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485a810-796d-4f3a-99fc-b6b20c88ed1b None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:57.909177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:58.006927 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485a810-796d-4f3a-99fc-b6b20c88ed1b None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:58.047189 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec0f9e-9768-4ed9-9d72-ebda0ee236b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:58.047884 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec0f9e-9768-4ed9-9d72-ebda0ee236b4 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:58.048239 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec0f9e-9768-4ed9-9d72-ebda0ee236b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:58.048637 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec0f9e-9768-4ed9-9d72-ebda0ee236b4 None admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:58.049636 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec0f9e-9768-4ed9-9d72-ebda0ee236b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:58.049934 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec0f9e-9768-4ed9-9d72-ebda0ee236b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:58.051610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec0f9e-9768-4ed9-9d72-ebda0ee236b4 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:58.071204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 647/1281] 10.222.0.93 () {56 vars in 1143 bytes} [Mon Aug 1 15:01:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:58.078411 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f4058-d8cf-43fe-9aee-0e7449584b03 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:58.194169 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:58.206328 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f4058-d8cf-43fe-9aee-0e7449584b03 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:58.248768 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31e972-dcc5-4fe3-83d6-23ab70737919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:58.249401 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31e972-dcc5-4fe3-83d6-23ab70737919 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:58.249706 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31e972-dcc5-4fe3-83d6-23ab70737919 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:58.249945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31e972-dcc5-4fe3-83d6-23ab70737919 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:58.334454 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31e972-dcc5-4fe3-83d6-23ab70737919 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:58.335909 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31e972-dcc5-4fe3-83d6-23ab70737919 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:58.426137 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:58.438213 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 635/1282] 10.222.0.93 () {58 vars in 1282 bytes} [Mon Aug 1 15:01:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 15:01:58.444340 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec0f9e-9768-4ed9-9d72-ebda0ee236b4 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:58.526024 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:58.568957 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec0f9e-9768-4ed9-9d72-ebda0ee236b4 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:58.605048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbdd7a-87ed-493f-b6e1-b685bcc3357f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:58.605597 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbdd7a-87ed-493f-b6e1-b685bcc3357f None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:58.605873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbdd7a-87ed-493f-b6e1-b685bcc3357f None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:58.606146 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbdd7a-87ed-493f-b6e1-b685bcc3357f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:58.618345 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbdd7a-87ed-493f-b6e1-b685bcc3357f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:58.619647 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbdd7a-87ed-493f-b6e1-b685bcc3357f None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:58.634906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-07dbdd7a-87ed-493f-b6e1-b685bcc3357f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:58.634906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:58.634906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:58.634906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:58.634906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:58.634906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:58.634906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:58.634906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:58.634906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:58.634906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:58.634906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:58.634906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:58.634906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:58.634906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:58.634906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:58.636064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:58.636064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:58.636064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:58.636064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:58.636064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:58.636064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:58.636064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:58.636064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:58.636064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:58.636064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:58.636064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:58.636064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:58.636064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:58.636064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:58.636064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:58.636064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:58.636064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:58.637414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:58.637414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:58.637414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:58.637414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:01:58.637414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 648/1283] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:58.642714 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31e972-dcc5-4fe3-83d6-23ab70737919 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:58.754831 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31e972-dcc5-4fe3-83d6-23ab70737919 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:58.791449 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde318f5-0ee8-4b3c-911b-fb54f03b24c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:58.792053 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde318f5-0ee8-4b3c-911b-fb54f03b24c6 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:58.792333 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde318f5-0ee8-4b3c-911b-fb54f03b24c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:58.792654 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde318f5-0ee8-4b3c-911b-fb54f03b24c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:58.793339 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde318f5-0ee8-4b3c-911b-fb54f03b24c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:58.793640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde318f5-0ee8-4b3c-911b-fb54f03b24c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:58.795029 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde318f5-0ee8-4b3c-911b-fb54f03b24c6 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:58.807578 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 636/1284] 10.222.0.93 () {56 vars in 1109 bytes} [Mon Aug 1 15:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:58.813971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbdd7a-87ed-493f-b6e1-b685bcc3357f None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:58.940516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbdd7a-87ed-493f-b6e1-b685bcc3357f None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:58.954864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:58.975918 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f7930-9095-4327-992f-40147f7327f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:58.976593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f7930-9095-4327-992f-40147f7327f7 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:58.976870 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f7930-9095-4327-992f-40147f7327f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:58.977140 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f7930-9095-4327-992f-40147f7327f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:58.984583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f7930-9095-4327-992f-40147f7327f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:58.986034 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f7930-9095-4327-992f-40147f7327f7 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:58.998473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application [None req-2c1f7930-9095-4327-992f-40147f7327f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:58.998473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:58.998473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:58.998473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:58.998473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:58.998473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:58.998473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:58.998473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:58.998473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:58.998473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:58.998473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:58.998473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:58.998473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:58.998473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:58.998473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:58.999645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:58.999645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:58.999645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:58.999645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:58.999645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:58.999645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:58.999645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:58.999645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:58.999645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:58.999645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:58.999645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:58.999645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:58.999645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:58.999645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:58.999645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:58.999645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:58.999645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:59.000879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:59.000879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:59.000879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:59.000879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:59.000879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:59.000879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:59.000879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:59.000879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: ERROR keystone.server.flask.application Aug 01 15:01:59.000879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 649/1285] 10.222.0.93 () {56 vars in 1111 bytes} [Mon Aug 1 15:01:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:59.005641 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde318f5-0ee8-4b3c-911b-fb54f03b24c6 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:59.061913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:59.112723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde318f5-0ee8-4b3c-911b-fb54f03b24c6 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:59.147264 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30b25c-7640-4eda-b2ff-5a1cbb4920cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:59.148203 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30b25c-7640-4eda-b2ff-5a1cbb4920cd None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:59.148203 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30b25c-7640-4eda-b2ff-5a1cbb4920cd None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:59.148429 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30b25c-7640-4eda-b2ff-5a1cbb4920cd None admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:59.149150 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30b25c-7640-4eda-b2ff-5a1cbb4920cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:59.149451 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30b25c-7640-4eda-b2ff-5a1cbb4920cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:59.150881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30b25c-7640-4eda-b2ff-5a1cbb4920cd None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:59.163325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 637/1286] 10.222.0.93 () {56 vars in 1148 bytes} [Mon Aug 1 15:01:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:59.170582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f7930-9095-4327-992f-40147f7327f7 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:59.275419 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f7930-9095-4327-992f-40147f7327f7 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:59.309538 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076657af-1d46-4921-a522-0f39622c3d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:59.310148 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076657af-1d46-4921-a522-0f39622c3d80 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:59.310430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076657af-1d46-4921-a522-0f39622c3d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:59.311264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076657af-1d46-4921-a522-0f39622c3d80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:59.312049 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076657af-1d46-4921-a522-0f39622c3d80 None admin] RBAC: Adding query filter params (role.id=5456887a306649bf80a9ea7bd4282e97, scope.project.id=ac9f7c74597642b0acecc956429ac4d8, user.id=a07e3e25e6a8466fb237dcb836aecda2) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:59.312359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076657af-1d46-4921-a522-0f39622c3d80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:59.313945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076657af-1d46-4921-a522-0f39622c3d80 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:59.320180 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 650/1287] 10.222.0.93 () {56 vars in 1374 bytes} [Mon Aug 1 15:01:59 2022] GET /identity/v3/role_assignments?user.id=a07e3e25e6a8466fb237dcb836aecda2&scope.project.id=ac9f7c74597642b0acecc956429ac4d8&role.id=5456887a306649bf80a9ea7bd4282e97 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:01.425713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37457cc-44ef-4751-8753-692e17eaf76f None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:01.426295 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37457cc-44ef-4751-8753-692e17eaf76f None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:01.426295 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37457cc-44ef-4751-8753-692e17eaf76f None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:01.427687 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 638/1288] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:02:01.433917 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a10078-f5f7-4117-9d41-4f4dace156a4 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:01.434702 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a10078-f5f7-4117-9d41-4f4dace156a4 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:01.434702 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a10078-f5f7-4117-9d41-4f4dace156a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:01.454861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-d2a10078-f5f7-4117-9d41-4f4dace156a4 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:02:01.456258 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-d2a10078-f5f7-4117-9d41-4f4dace156a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:02:01.491310 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:01.537729 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 651/1289] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:02:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:02:01.562546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37457cc-44ef-4751-8753-692e17eaf76f None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:01.675289 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37457cc-44ef-4751-8753-692e17eaf76f None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:01.687515 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:01.714127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e31c1-fd77-46fa-8931-84c12d3067c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:01.714766 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e31c1-fd77-46fa-8931-84c12d3067c2 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:01.715045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e31c1-fd77-46fa-8931-84c12d3067c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:01.715303 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e31c1-fd77-46fa-8931-84c12d3067c2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:01.716407 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e31c1-fd77-46fa-8931-84c12d3067c2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:01.719572 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e31c1-fd77-46fa-8931-84c12d3067c2 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:01.736490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-d14e31c1-fd77-46fa-8931-84c12d3067c2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:02:01.736490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:02:01.736490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:02:01.736490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:02:01.736490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:02:01.736490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:02:01.736490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:02:01.736490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:02:01.736490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:02:01.736490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:02:01.736490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:02:01.736490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:02:01.736490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:02:01.736490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:02:01.736490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:02:01.736490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:02:01.736490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:02:01.737843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:02:01.737843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:02:01.737843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:02:01.737843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:02:01.737843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:02:01.737843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:02:01.737843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:02:01.737843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:02:01.737843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:02:01.737843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:02:01.737843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:02:01.737843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:02:01.737843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:02:01.737843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:02:01.737843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:02:01.737843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:02:01.739060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:02:01.739060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:02:01.739060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:02:01.739060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:02:01.739060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:02:01.739060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:02:01.739060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:02:01.739060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:02:01.739060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:02:01.739060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:02:01.739060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:02:01.739060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:02:01.739060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:02:01.739060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:02:01.739060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 639/1290] 10.222.0.93 () {56 vars in 1117 bytes} [Mon Aug 1 15:02:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:02:01.745537 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a10078-f5f7-4117-9d41-4f4dace156a4 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:01.866689 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a10078-f5f7-4117-9d41-4f4dace156a4 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:01.921914 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59a3a6-e1f2-4702-93ad-74e9c2b7ce64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:01.922589 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59a3a6-e1f2-4702-93ad-74e9c2b7ce64 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:01.922871 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59a3a6-e1f2-4702-93ad-74e9c2b7ce64 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:01.923145 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59a3a6-e1f2-4702-93ad-74e9c2b7ce64 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:01.923844 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59a3a6-e1f2-4702-93ad-74e9c2b7ce64 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:02:01.924148 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59a3a6-e1f2-4702-93ad-74e9c2b7ce64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:01.925631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59a3a6-e1f2-4702-93ad-74e9c2b7ce64 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:01.931693 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 652/1291] 10.222.0.93 () {56 vars in 1116 bytes} [Mon Aug 1 15:02:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:01.938958 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e31c1-fd77-46fa-8931-84c12d3067c2 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:02.108070 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e31c1-fd77-46fa-8931-84c12d3067c2 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:02.152859 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d4025-d5b9-4ee4-9c38-dc228f553d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:02.153769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d4025-d5b9-4ee4-9c38-dc228f553d84 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:02.154076 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d4025-d5b9-4ee4-9c38-dc228f553d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:02.154467 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d4025-d5b9-4ee4-9c38-dc228f553d84 None admin] PATH_INFO: `/v3/services` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:02.155290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d4025-d5b9-4ee4-9c38-dc228f553d84 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:02:02.155676 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d4025-d5b9-4ee4-9c38-dc228f553d84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:02.157287 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d4025-d5b9-4ee4-9c38-dc228f553d84 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:02.162381 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 640/1292] 10.222.0.93 () {56 vars in 1116 bytes} [Mon Aug 1 15:02:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:02:04.222804 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaee4d5-f861-41b5-8e7c-c01bfc0f3388 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:04.223378 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaee4d5-f861-41b5-8e7c-c01bfc0f3388 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:04.223775 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaee4d5-f861-41b5-8e7c-c01bfc0f3388 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:04.226466 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 653/1293] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:02:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:02:04.233307 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a4153-997a-4ec2-9e95-5fc2e3a96cf6 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:04.233911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a4153-997a-4ec2-9e95-5fc2e3a96cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:04.233911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a4153-997a-4ec2-9e95-5fc2e3a96cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:04.258473 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-902a4153-997a-4ec2-9e95-5fc2e3a96cf6 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:02:04.259742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-902a4153-997a-4ec2-9e95-5fc2e3a96cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:02:04.297588 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:04.343266 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 641/1294] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:02:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:02:04.366430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaee4d5-f861-41b5-8e7c-c01bfc0f3388 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:04.402417 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:04.483219 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaee4d5-f861-41b5-8e7c-c01bfc0f3388 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:04.526133 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929f627-8b45-44e7-b9a5-6096095a5b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:04.526759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929f627-8b45-44e7-b9a5-6096095a5b50 None admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:04.527037 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929f627-8b45-44e7-b9a5-6096095a5b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:04.527317 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929f627-8b45-44e7-b9a5-6096095a5b50 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:04.528019 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929f627-8b45-44e7-b9a5-6096095a5b50 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:04.529475 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929f627-8b45-44e7-b9a5-6096095a5b50 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:04.545995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 654/1295] 10.222.0.93 () {60 vars in 1141 bytes} [Mon Aug 1 15:02:04 2022] POST /identity/v3/services => generated 246 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:02:06.623228 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fd1ba-949c-4477-979e-fda4b09da83c None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:06.623776 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fd1ba-949c-4477-979e-fda4b09da83c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:06.623776 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fd1ba-949c-4477-979e-fda4b09da83c None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:06.625200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 642/1296] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:02:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:02:06.630883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dd070-f16e-4397-b5ac-352c10c8187b None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:06.631376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dd070-f16e-4397-b5ac-352c10c8187b None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:06.631464 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dd070-f16e-4397-b5ac-352c10c8187b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:06.650685 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-184dd070-f16e-4397-b5ac-352c10c8187b None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:02:06.651890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-184dd070-f16e-4397-b5ac-352c10c8187b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:02:06.742179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 655/1297] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:02:06 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:02:06.765555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fd1ba-949c-4477-979e-fda4b09da83c None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:06.894438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fd1ba-949c-4477-979e-fda4b09da83c None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:06.938950 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c050fe5-b2c5-41c9-b9f0-308c53c20a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:06.939606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c050fe5-b2c5-41c9-b9f0-308c53c20a46 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:06.939888 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c050fe5-b2c5-41c9-b9f0-308c53c20a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:06.940162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c050fe5-b2c5-41c9-b9f0-308c53c20a46 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:06.940844 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c050fe5-b2c5-41c9-b9f0-308c53c20a46 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:06.942243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c050fe5-b2c5-41c9-b9f0-308c53c20a46 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:06.952199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-1c050fe5-b2c5-41c9-b9f0-308c53c20a46 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:02:06.952199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:02:06.952199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:02:06.952199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:02:06.952199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:02:06.952199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:02:06.952199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:02:06.952199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:02:06.952199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:02:06.952199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:02:06.952199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:02:06.952199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:02:06.952199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:02:06.952199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:02:06.952199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:02:06.952199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:02:06.952199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:02:06.953464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:02:06.953464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:02:06.953464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:02:06.953464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:02:06.953464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:02:06.953464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:02:06.953464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:02:06.953464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:02:06.953464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:02:06.953464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:02:06.953464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:02:06.953464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:02:06.953464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:02:06.953464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:02:06.953464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:02:06.953464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:02:06.954478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:02:06.954478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:02:06.954478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:02:06.954478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:02:06.954478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:02:06.954478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:02:06.954478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:02:06.954478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:02:06.954478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:02:06.954478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:02:06.954478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:02:06.954478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:02:06.954478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:02:06.954478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:02:06.954478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 643/1298] 10.222.0.93 () {56 vars in 1117 bytes} [Mon Aug 1 15:02:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:02:06.960010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184dd070-f16e-4397-b5ac-352c10c8187b None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:06.979897 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:07.088885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184dd070-f16e-4397-b5ac-352c10c8187b None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:07.136522 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e05bf6-c070-46bf-bb98-6c5ab6f49a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:07.137170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e05bf6-c070-46bf-bb98-6c5ab6f49a15 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:07.137583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e05bf6-c070-46bf-bb98-6c5ab6f49a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:07.137824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e05bf6-c070-46bf-bb98-6c5ab6f49a15 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:07.139115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e05bf6-c070-46bf-bb98-6c5ab6f49a15 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:02:07.139415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e05bf6-c070-46bf-bb98-6c5ab6f49a15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:07.140929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e05bf6-c070-46bf-bb98-6c5ab6f49a15 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:07.147147 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 656/1299] 10.222.0.93 () {56 vars in 1116 bytes} [Mon Aug 1 15:02:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:07.154403 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c050fe5-b2c5-41c9-b9f0-308c53c20a46 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:07.189950 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:07.293742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c050fe5-b2c5-41c9-b9f0-308c53c20a46 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:07.332863 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b16e1-61bd-4e4b-b081-06ea1fad065d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:07.333527 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b16e1-61bd-4e4b-b081-06ea1fad065d None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:07.333796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b16e1-61bd-4e4b-b081-06ea1fad065d None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:07.334150 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b16e1-61bd-4e4b-b081-06ea1fad065d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:07.335120 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b16e1-61bd-4e4b-b081-06ea1fad065d None admin] RBAC: Adding query filter params (interface=public, service_id=c0f968f648ee4656a7c0a60158877588, region_id=RegionOne) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:02:07.335488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b16e1-61bd-4e4b-b081-06ea1fad065d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:07.337201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b16e1-61bd-4e4b-b081-06ea1fad065d None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:07.344238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 644/1300] 10.222.0.93 () {56 vars in 1251 bytes} [Mon Aug 1 15:02:07 2022] GET /identity/v3/endpoints?service_id=c0f968f648ee4656a7c0a60158877588&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:07.350660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e05bf6-c070-46bf-bb98-6c5ab6f49a15 None admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:07.454521 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:07.479106 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e05bf6-c070-46bf-bb98-6c5ab6f49a15 None admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:07.542220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910609b1-dba6-46c5-81c4-c4b2c3aeaade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:07.543297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910609b1-dba6-46c5-81c4-c4b2c3aeaade None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:07.543780 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910609b1-dba6-46c5-81c4-c4b2c3aeaade None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:07.544114 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910609b1-dba6-46c5-81c4-c4b2c3aeaade None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:07.545951 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910609b1-dba6-46c5-81c4-c4b2c3aeaade None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:07.548650 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910609b1-dba6-46c5-81c4-c4b2c3aeaade None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:07.563767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 657/1301] 10.222.0.93 () {56 vars in 1087 bytes} [Mon Aug 1 15:02:07 2022] GET /identity/v3/services => generated 2191 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:02:09.647289 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acba64-4ba0-446e-8aed-1a68a50a897c None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:09.647839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acba64-4ba0-446e-8aed-1a68a50a897c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:09.647839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acba64-4ba0-446e-8aed-1a68a50a897c None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:09.649216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 645/1302] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:02:09.655511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b8ca3-20d2-4a91-9d47-e3d3914015d7 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:09.655980 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b8ca3-20d2-4a91-9d47-e3d3914015d7 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:09.656067 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b8ca3-20d2-4a91-9d47-e3d3914015d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:09.674271 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-3c3b8ca3-20d2-4a91-9d47-e3d3914015d7 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:02:09.675478 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3c3b8ca3-20d2-4a91-9d47-e3d3914015d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:02:09.746590 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 658/1303] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:02:09 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:02:09.770164 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40acba64-4ba0-446e-8aed-1a68a50a897c None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:09.883019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:09.891551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40acba64-4ba0-446e-8aed-1a68a50a897c None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:09.926887 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a1f98-9b52-45af-a103-bfcb31430741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:09.927528 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a1f98-9b52-45af-a103-bfcb31430741 None admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:09.927811 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a1f98-9b52-45af-a103-bfcb31430741 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:09.928076 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a1f98-9b52-45af-a103-bfcb31430741 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:09.928727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a1f98-9b52-45af-a103-bfcb31430741 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:09.930117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a1f98-9b52-45af-a103-bfcb31430741 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:09.939678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application [None req-464a1f98-9b52-45af-a103-bfcb31430741 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:02:09.939678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:02:09.939678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:02:09.939678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:02:09.939678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:02:09.939678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:02:09.939678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:02:09.939678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:02:09.939678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:02:09.939678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:02:09.939678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:02:09.939678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:02:09.939678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:02:09.939678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:02:09.939678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:02:09.939678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:02:09.939678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:02:09.940985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:02:09.940985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:02:09.940985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:02:09.940985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:02:09.940985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:02:09.940985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:02:09.940985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:02:09.940985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:02:09.940985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:02:09.940985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:02:09.940985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:02:09.940985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:02:09.940985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:02:09.940985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:02:09.940985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:02:09.940985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:02:09.942156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:02:09.942156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application with Lock( Aug 01 15:02:09.942156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:02:09.942156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:02:09.942156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:02:09.942156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:02:09.942156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:02:09.942156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:02:09.942156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:02:09.942156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:02:09.942156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:02:09.942156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:02:09.942156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:02:09.942156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: ERROR keystone.server.flask.application Aug 01 15:02:09.942156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 646/1304] 10.222.0.93 () {56 vars in 1117 bytes} [Mon Aug 1 15:02:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:02:09.947668 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b8ca3-20d2-4a91-9d47-e3d3914015d7 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:10.037874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:10.065567 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b8ca3-20d2-4a91-9d47-e3d3914015d7 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:10.104162 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38088bb3-3ce0-440d-afa0-5674972c2240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:10.104814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38088bb3-3ce0-440d-afa0-5674972c2240 None admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:10.105084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38088bb3-3ce0-440d-afa0-5674972c2240 None admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:10.105349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38088bb3-3ce0-440d-afa0-5674972c2240 None admin] PATH_INFO: `/v3/services` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:10.106026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38088bb3-3ce0-440d-afa0-5674972c2240 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:02:10.106317 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38088bb3-3ce0-440d-afa0-5674972c2240 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:10.107692 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38088bb3-3ce0-440d-afa0-5674972c2240 None admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:10.113239 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 659/1305] 10.222.0.93 () {56 vars in 1116 bytes} [Mon Aug 1 15:02:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:10.119897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a1f98-9b52-45af-a103-bfcb31430741 None admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:10.225520 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a1f98-9b52-45af-a103-bfcb31430741 None admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:10.260231 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518b353-5098-42d6-afcc-1c4438d98994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:10.260877 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518b353-5098-42d6-afcc-1c4438d98994 None admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:10.261161 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518b353-5098-42d6-afcc-1c4438d98994 None admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:10.261438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518b353-5098-42d6-afcc-1c4438d98994 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:10.262155 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518b353-5098-42d6-afcc-1c4438d98994 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:10.264218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518b353-5098-42d6-afcc-1c4438d98994 None admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:10.285055 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 647/1306] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:02:10 2022] POST /identity/v3/endpoints => generated 331 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:02:20.111343 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=30 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:20.281234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:19.798916 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b5652-d7a2-4672-be78-dd508d9cb5dc None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:19.799623 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b5652-d7a2-4672-be78-dd508d9cb5dc None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:19.799780 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b5652-d7a2-4672-be78-dd508d9cb5dc None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:19.803921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 660/1307] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:03:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:03:19.809984 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2861a-9bba-47e5-ac51-4f54e77da7d1 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:19.810895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2861a-9bba-47e5-ac51-4f54e77da7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:19.811331 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2861a-9bba-47e5-ac51-4f54e77da7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:19.927766 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-22f2861a-9bba-47e5-ac51-4f54e77da7d1 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:03:19.929154 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-22f2861a-9bba-47e5-ac51-4f54e77da7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:20.049984 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 648/1308] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:03:20.102320 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c1739-c2e6-4868-99fb-6ca3f19570ce None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:20.102601 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c1739-c2e6-4868-99fb-6ca3f19570ce None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:20.103123 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c1739-c2e6-4868-99fb-6ca3f19570ce None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:20.104532 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 661/1309] 10.222.0.93 () {54 vars in 965 bytes} [Mon Aug 1 15:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:03:20.112163 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09928c-1977-458d-b625-19255c8bc958 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:20.112815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09928c-1977-458d-b625-19255c8bc958 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:20.113272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09928c-1977-458d-b625-19255c8bc958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:20.253888 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-ad09928c-1977-458d-b625-19255c8bc958 None None] MFA Rules not processed for user `4eba5f64833b482d81c53721c8442970`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:03:20.255980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ad09928c-1977-458d-b625-19255c8bc958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:20.287118 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:20.374729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 649/1310] 10.222.0.93 () {58 vars in 1062 bytes} [Mon Aug 1 15:03:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:03:20.383630 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c1739-c2e6-4868-99fb-6ca3f19570ce None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:20.385928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-c83c1739-c2e6-4868-99fb-6ca3f19570ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:20.630958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c1739-c2e6-4868-99fb-6ca3f19570ce None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:20.798493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b58f1-98fc-4c87-aa62-e90454386e2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:20.799512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b58f1-98fc-4c87-aa62-e90454386e2e service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:20.800095 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b58f1-98fc-4c87-aa62-e90454386e2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:20.800526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b58f1-98fc-4c87-aa62-e90454386e2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:20.804139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-2d8b58f1-98fc-4c87-aa62-e90454386e2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:20.900114 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:20.976038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b58f1-98fc-4c87-aa62-e90454386e2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:20.977547 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b58f1-98fc-4c87-aa62-e90454386e2e service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:21.012108 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 662/1311] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:03:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:03:23.609840 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeef81c-6ab4-4a9d-b44b-b4648ddaaef7 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:23.610632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeef81c-6ab4-4a9d-b44b-b4648ddaaef7 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:23.610632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeef81c-6ab4-4a9d-b44b-b4648ddaaef7 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:23.611744 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 650/1312] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:03:23.617840 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72a246-9976-44b4-af84-04371adc709f None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:23.618401 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72a246-9976-44b4-af84-04371adc709f None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:23.618401 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72a246-9976-44b4-af84-04371adc709f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:23.727400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-2a72a246-9976-44b4-af84-04371adc709f None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:03:23.728699 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-2a72a246-9976-44b4-af84-04371adc709f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:23.841169 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 663/1313] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:03:23.897537 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fc799-28ab-43b6-8cd2-04d01f04af10 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:23.897821 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fc799-28ab-43b6-8cd2-04d01f04af10 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:23.898147 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fc799-28ab-43b6-8cd2-04d01f04af10 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:23.899633 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 651/1314] 10.222.0.93 () {54 vars in 965 bytes} [Mon Aug 1 15:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:03:23.906815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e62e3-e940-4af5-8af9-db28f4e9b81a None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:23.907284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e62e3-e940-4af5-8af9-db28f4e9b81a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:23.907378 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e62e3-e940-4af5-8af9-db28f4e9b81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:23.920089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:24.002797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-882e62e3-e940-4af5-8af9-db28f4e9b81a None None] MFA Rules not processed for user `4eba5f64833b482d81c53721c8442970`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:03:24.004195 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-882e62e3-e940-4af5-8af9-db28f4e9b81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:24.108335 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 664/1315] 10.222.0.93 () {58 vars in 1062 bytes} [Mon Aug 1 15:03:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:03:24.119635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fc799-28ab-43b6-8cd2-04d01f04af10 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:24.122520 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-299fc799-28ab-43b6-8cd2-04d01f04af10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:24.264925 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fc799-28ab-43b6-8cd2-04d01f04af10 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:24.345030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:24.347999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e7e64-0e48-4558-8d89-3100b9c77b2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:24.348879 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e7e64-0e48-4558-8d89-3100b9c77b2a service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:24.349222 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e7e64-0e48-4558-8d89-3100b9c77b2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:24.349645 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e7e64-0e48-4558-8d89-3100b9c77b2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:24.351619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-762e7e64-0e48-4558-8d89-3100b9c77b2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:24.480485 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e7e64-0e48-4558-8d89-3100b9c77b2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:24.482105 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e7e64-0e48-4558-8d89-3100b9c77b2a service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:24.506192 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 652/1316] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:03:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:03:27.196994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5f46a-2a39-41e7-9a87-478b0ffec53a None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:27.197571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5f46a-2a39-41e7-9a87-478b0ffec53a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:27.197571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5f46a-2a39-41e7-9a87-478b0ffec53a None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:27.199442 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 665/1317] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:03:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:03:27.204788 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49be1f-fa89-4904-9299-e6d2b993d6e5 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:27.205242 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49be1f-fa89-4904-9299-e6d2b993d6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:27.205368 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49be1f-fa89-4904-9299-e6d2b993d6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:27.271951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-5e49be1f-fa89-4904-9299-e6d2b993d6e5 None None] MFA Rules not processed for user `d87f64f8b42c4884ba624a882478570d`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:03:27.273208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-5e49be1f-fa89-4904-9299-e6d2b993d6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:27.326245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:27.374255 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 653/1318] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:03:27 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:03:27.414427 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5f46a-2a39-41e7-9a87-478b0ffec53a None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:27.415974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-6ed5f46a-2a39-41e7-9a87-478b0ffec53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:27.524820 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5f46a-2a39-41e7-9a87-478b0ffec53a None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:27.528106 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:27.604037 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c746d0a-2063-4aec-b9b5-0f055656a954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:27.604670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c746d0a-2063-4aec-b9b5-0f055656a954 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:27.604950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c746d0a-2063-4aec-b9b5-0f055656a954 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:27.605220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c746d0a-2063-4aec-b9b5-0f055656a954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:27.606890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-7c746d0a-2063-4aec-b9b5-0f055656a954 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:27.722050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c746d0a-2063-4aec-b9b5-0f055656a954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:27.723954 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c746d0a-2063-4aec-b9b5-0f055656a954 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:27.744951 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 666/1319] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:03:27 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:03:30.529884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83936d52-c620-426f-a339-0c8d36fc6543 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:30.530475 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83936d52-c620-426f-a339-0c8d36fc6543 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:30.530475 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83936d52-c620-426f-a339-0c8d36fc6543 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:30.531969 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 654/1320] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:03:30.537857 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609e94e-74de-4a3f-8538-40277822e7e1 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:30.538373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609e94e-74de-4a3f-8538-40277822e7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:30.538464 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609e94e-74de-4a3f-8538-40277822e7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:30.582328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:30.608600 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-7609e94e-74de-4a3f-8538-40277822e7e1 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:03:30.609862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-7609e94e-74de-4a3f-8538-40277822e7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:30.709325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 667/1321] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:03:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:03:30.750098 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83936d52-c620-426f-a339-0c8d36fc6543 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:30.751654 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-83936d52-c620-426f-a339-0c8d36fc6543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:30.879243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83936d52-c620-426f-a339-0c8d36fc6543 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:30.962282 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ec5f1-0c54-4ef2-b341-954dddbef26b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:30.962904 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ec5f1-0c54-4ef2-b341-954dddbef26b service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:30.963181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ec5f1-0c54-4ef2-b341-954dddbef26b service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:30.963453 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ec5f1-0c54-4ef2-b341-954dddbef26b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:30.965167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-cc9ec5f1-0c54-4ef2-b341-954dddbef26b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:30.972509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:31.084280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ec5f1-0c54-4ef2-b341-954dddbef26b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:31.085770 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ec5f1-0c54-4ef2-b341-954dddbef26b service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:31.107535 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 655/1322] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:03:30 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:03:34.011754 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486056f3-88f6-4d9f-8273-a9513f571c4e None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:34.012345 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486056f3-88f6-4d9f-8273-a9513f571c4e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:34.012345 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486056f3-88f6-4d9f-8273-a9513f571c4e None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:34.013760 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 668/1323] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:03:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:03:34.019303 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b00972-aa48-4154-8c1b-eac51d069a24 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:34.019805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b00972-aa48-4154-8c1b-eac51d069a24 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:34.019895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b00972-aa48-4154-8c1b-eac51d069a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:34.094998 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-b7b00972-aa48-4154-8c1b-eac51d069a24 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:03:34.096232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b7b00972-aa48-4154-8c1b-eac51d069a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:34.171651 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:34.238313 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 656/1324] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:03:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:03:34.277551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486056f3-88f6-4d9f-8273-a9513f571c4e None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:34.279235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-486056f3-88f6-4d9f-8273-a9513f571c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:34.445323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486056f3-88f6-4d9f-8273-a9513f571c4e None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:34.467870 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:34.546001 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8d810-2c4d-46d7-89dc-a0893d88675d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:34.546621 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8d810-2c4d-46d7-89dc-a0893d88675d service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:34.546895 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8d810-2c4d-46d7-89dc-a0893d88675d service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:34.547162 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8d810-2c4d-46d7-89dc-a0893d88675d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:34.548870 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-51f8d810-2c4d-46d7-89dc-a0893d88675d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:34.671389 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8d810-2c4d-46d7-89dc-a0893d88675d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:34.672804 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8d810-2c4d-46d7-89dc-a0893d88675d service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:34.693311 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 669/1325] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:03:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:03:36.879175 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dee67-5f69-47f8-828e-672d4b28452b None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:36.879930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dee67-5f69-47f8-828e-672d4b28452b None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:36.879930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dee67-5f69-47f8-828e-672d4b28452b None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:36.881331 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 657/1326] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:03:36.887256 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012beb28-fed1-4c6b-87cd-4b5190289a35 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:36.887888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012beb28-fed1-4c6b-87cd-4b5190289a35 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:36.887888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012beb28-fed1-4c6b-87cd-4b5190289a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:36.947547 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:36.969421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-012beb28-fed1-4c6b-87cd-4b5190289a35 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:03:36.970646 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-012beb28-fed1-4c6b-87cd-4b5190289a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:37.091439 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 670/1327] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:03:37.134015 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dee67-5f69-47f8-828e-672d4b28452b None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:37.136143 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-6c6dee67-5f69-47f8-828e-672d4b28452b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:37.287558 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dee67-5f69-47f8-828e-672d4b28452b None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:37.406577 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97920867-6809-4f52-b120-aca8d5388fc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:37.407197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97920867-6809-4f52-b120-aca8d5388fc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:37.407635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97920867-6809-4f52-b120-aca8d5388fc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:37.408115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97920867-6809-4f52-b120-aca8d5388fc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:37.410060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-97920867-6809-4f52-b120-aca8d5388fc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:37.432332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:37.581673 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97920867-6809-4f52-b120-aca8d5388fc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:37.583172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97920867-6809-4f52-b120-aca8d5388fc6 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:37.608227 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 658/1328] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:03:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:03:39.764388 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ceaef6-5263-4cd7-8f6b-f4864f7054bc None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:39.764941 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ceaef6-5263-4cd7-8f6b-f4864f7054bc None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:39.764941 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ceaef6-5263-4cd7-8f6b-f4864f7054bc None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:39.766286 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 671/1329] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:03:39.772328 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5e331-cc7f-4fbd-b34b-2da80f81197c None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:39.772910 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5e331-cc7f-4fbd-b34b-2da80f81197c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:39.773178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5e331-cc7f-4fbd-b34b-2da80f81197c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:39.857518 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-c7b5e331-cc7f-4fbd-b34b-2da80f81197c None None] MFA Rules not processed for user `d87f64f8b42c4884ba624a882478570d`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:03:39.858744 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c7b5e331-cc7f-4fbd-b34b-2da80f81197c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:39.941555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:39.980887 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 659/1330] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:03:39 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:03:40.022391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ceaef6-5263-4cd7-8f6b-f4864f7054bc None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:40.031342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-11ceaef6-5263-4cd7-8f6b-f4864f7054bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:40.154523 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ceaef6-5263-4cd7-8f6b-f4864f7054bc None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:40.185082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:40.242904 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857943e9-e662-44a2-a366-c11a359d879c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:40.243516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857943e9-e662-44a2-a366-c11a359d879c service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:40.243982 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857943e9-e662-44a2-a366-c11a359d879c service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:40.244088 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857943e9-e662-44a2-a366-c11a359d879c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:40.245761 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-857943e9-e662-44a2-a366-c11a359d879c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:40.396286 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857943e9-e662-44a2-a366-c11a359d879c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:40.398081 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857943e9-e662-44a2-a366-c11a359d879c service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:40.421452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 672/1331] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:03:40 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:03:42.915786 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f50ae-5d1a-4687-8a9d-95f6d7990960 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:42.916564 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f50ae-5d1a-4687-8a9d-95f6d7990960 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:42.916564 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f50ae-5d1a-4687-8a9d-95f6d7990960 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:42.918404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 660/1332] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:03:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:03:42.924934 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51354777-1288-4f45-88ac-26a059821cdd None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:42.925922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51354777-1288-4f45-88ac-26a059821cdd None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:42.925922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51354777-1288-4f45-88ac-26a059821cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:43.002586 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:43.007326 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-51354777-1288-4f45-88ac-26a059821cdd None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:03:43.009952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-51354777-1288-4f45-88ac-26a059821cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:43.127115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 673/1333] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:03:43.164396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f50ae-5d1a-4687-8a9d-95f6d7990960 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:43.166064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-121f50ae-5d1a-4687-8a9d-95f6d7990960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:43.292034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f50ae-5d1a-4687-8a9d-95f6d7990960 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:43.377453 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917f465-06b2-430b-805a-9ac2945319bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:43.378079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917f465-06b2-430b-805a-9ac2945319bd service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:43.378448 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917f465-06b2-430b-805a-9ac2945319bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:43.378637 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917f465-06b2-430b-805a-9ac2945319bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:43.380353 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a917f465-06b2-430b-805a-9ac2945319bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:43.413558 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:43.513855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917f465-06b2-430b-805a-9ac2945319bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:43.515331 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917f465-06b2-430b-805a-9ac2945319bd service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:43.536044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 661/1334] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:03:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:03:45.634107 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895c330-8692-41bb-b2b0-b800dd0d559e None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:45.634712 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895c330-8692-41bb-b2b0-b800dd0d559e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:45.634712 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895c330-8692-41bb-b2b0-b800dd0d559e None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:45.636767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 674/1335] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:03:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:03:45.642023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f10836-774c-4e38-9929-5e47ad8d0b70 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:45.642836 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f10836-774c-4e38-9929-5e47ad8d0b70 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:45.643115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f10836-774c-4e38-9929-5e47ad8d0b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:45.708692 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-53f10836-774c-4e38-9929-5e47ad8d0b70 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:03:45.709915 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-53f10836-774c-4e38-9929-5e47ad8d0b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:45.786321 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:45.810157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 662/1336] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:03:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:03:45.853143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895c330-8692-41bb-b2b0-b800dd0d559e None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:45.855154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e895c330-8692-41bb-b2b0-b800dd0d559e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:45.964225 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895c330-8692-41bb-b2b0-b800dd0d559e None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:45.992716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:46.039047 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b6582-edaf-4436-85aa-9424e1fbabe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:46.039576 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b6582-edaf-4436-85aa-9424e1fbabe3 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:46.039853 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b6582-edaf-4436-85aa-9424e1fbabe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:46.040120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b6582-edaf-4436-85aa-9424e1fbabe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:46.041781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-388b6582-edaf-4436-85aa-9424e1fbabe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:46.153098 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b6582-edaf-4436-85aa-9424e1fbabe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:46.154503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b6582-edaf-4436-85aa-9424e1fbabe3 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:46.173107 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 675/1337] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:03:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:03:48.620597 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b62632-5400-4d8c-bd87-9282783aaf80 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:48.621232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b62632-5400-4d8c-bd87-9282783aaf80 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:48.621232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b62632-5400-4d8c-bd87-9282783aaf80 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:48.622526 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 663/1338] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:03:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:03:48.628731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee6e02-dea0-4455-8657-786042fec377 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:48.629386 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee6e02-dea0-4455-8657-786042fec377 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:48.629386 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee6e02-dea0-4455-8657-786042fec377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:48.698695 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-5eee6e02-dea0-4455-8657-786042fec377 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:03:48.699908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5eee6e02-dea0-4455-8657-786042fec377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:48.708707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:48.802072 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 676/1339] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:03:48 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:03:48.844693 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b62632-5400-4d8c-bd87-9282783aaf80 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:48.846276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c8b62632-5400-4d8c-bd87-9282783aaf80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:48.955676 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b62632-5400-4d8c-bd87-9282783aaf80 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:49.034373 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e3340-16ac-442b-9e12-ceb7a98eee91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:49.034994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e3340-16ac-442b-9e12-ceb7a98eee91 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:49.035277 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e3340-16ac-442b-9e12-ceb7a98eee91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:49.035550 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e3340-16ac-442b-9e12-ceb7a98eee91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:49.037239 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e58e3340-16ac-442b-9e12-ceb7a98eee91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:49.082442 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:49.169238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e3340-16ac-442b-9e12-ceb7a98eee91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:49.170636 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e3340-16ac-442b-9e12-ceb7a98eee91 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:49.193085 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 664/1340] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:03:48 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:03:53.290181 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37faccee-61f0-4364-974b-4c299c0169e4 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:53.290739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37faccee-61f0-4364-974b-4c299c0169e4 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:53.290941 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37faccee-61f0-4364-974b-4c299c0169e4 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:53.292795 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 677/1341] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:03:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:03:53.299377 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32284d2d-729d-4207-9ad9-bd2ff4503423 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:53.299922 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32284d2d-729d-4207-9ad9-bd2ff4503423 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:53.299922 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32284d2d-729d-4207-9ad9-bd2ff4503423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:53.372106 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-32284d2d-729d-4207-9ad9-bd2ff4503423 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:03:53.373364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-32284d2d-729d-4207-9ad9-bd2ff4503423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:53.475499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:53.490100 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 665/1342] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:03:53.532276 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37faccee-61f0-4364-974b-4c299c0169e4 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:53.535278 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-37faccee-61f0-4364-974b-4c299c0169e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:53.663293 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37faccee-61f0-4364-974b-4c299c0169e4 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:53.705386 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:53.748172 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f5348-a26a-4042-9e83-a8d588733aae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:53.748774 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f5348-a26a-4042-9e83-a8d588733aae service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:53.749057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f5348-a26a-4042-9e83-a8d588733aae service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:53.749323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f5348-a26a-4042-9e83-a8d588733aae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:53.751011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-369f5348-a26a-4042-9e83-a8d588733aae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:53.882987 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f5348-a26a-4042-9e83-a8d588733aae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:53.884456 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f5348-a26a-4042-9e83-a8d588733aae service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:53.904414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 678/1343] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:03:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:03:56.571738 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cabb2a-e1fd-4424-8af6-406d5d381d73 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:56.572316 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cabb2a-e1fd-4424-8af6-406d5d381d73 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:56.572316 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cabb2a-e1fd-4424-8af6-406d5d381d73 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:56.574455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 666/1344] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:03:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:03:56.580318 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bec73-0b34-4b4e-828e-e856774403fb None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:56.580867 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bec73-0b34-4b4e-828e-e856774403fb None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:56.580981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bec73-0b34-4b4e-828e-e856774403fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:56.655511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-393bec73-0b34-4b4e-828e-e856774403fb None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:03:56.656762 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-393bec73-0b34-4b4e-828e-e856774403fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:56.679377 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:56.763876 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 679/1345] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:03:56 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:03:56.802000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cabb2a-e1fd-4424-8af6-406d5d381d73 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:56.803534 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d6cabb2a-e1fd-4424-8af6-406d5d381d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:56.923499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cabb2a-e1fd-4424-8af6-406d5d381d73 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:57.001846 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd9f5a-f9d5-4809-9464-64499e6856e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:57.002461 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd9f5a-f9d5-4809-9464-64499e6856e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:57.002741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd9f5a-f9d5-4809-9464-64499e6856e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:57.003015 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd9f5a-f9d5-4809-9464-64499e6856e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:57.004679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b2dd9f5a-f9d5-4809-9464-64499e6856e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:57.050728 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:57.121913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd9f5a-f9d5-4809-9464-64499e6856e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:57.123354 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd9f5a-f9d5-4809-9464-64499e6856e9 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:57.142938 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 667/1346] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:03:56 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:04:01.319510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d328e2-b4dd-45fd-a180-9e83d20c607c None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:01.320078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d328e2-b4dd-45fd-a180-9e83d20c607c None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:01.320078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d328e2-b4dd-45fd-a180-9e83d20c607c None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:01.321815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 680/1347] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:01.327994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff02c5-66e8-4ac4-ba95-828ad1a91778 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:01.328561 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff02c5-66e8-4ac4-ba95-828ad1a91778 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:01.328661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff02c5-66e8-4ac4-ba95-828ad1a91778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:01.405512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-f9ff02c5-66e8-4ac4-ba95-828ad1a91778 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:01.406811 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-f9ff02c5-66e8-4ac4-ba95-828ad1a91778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:01.538573 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:01.552758 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 668/1348] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:04:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:04:01.581006 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d328e2-b4dd-45fd-a180-9e83d20c607c None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:01.582534 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-75d328e2-b4dd-45fd-a180-9e83d20c607c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:01.745599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d328e2-b4dd-45fd-a180-9e83d20c607c None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:01.805042 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:01.840069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc1084-4644-4945-9143-28aad532fa20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:01.840723 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc1084-4644-4945-9143-28aad532fa20 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:01.841002 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc1084-4644-4945-9143-28aad532fa20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:01.841271 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc1084-4644-4945-9143-28aad532fa20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:01.844081 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-d3cc1084-4644-4945-9143-28aad532fa20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:01.998466 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc1084-4644-4945-9143-28aad532fa20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:02.000029 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc1084-4644-4945-9143-28aad532fa20 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:02.032091 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 681/1349] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:04:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:04:07.465339 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d1661-5dce-4756-8c10-bd2442327fa2 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:07.466024 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d1661-5dce-4756-8c10-bd2442327fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:07.467275 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d1661-5dce-4756-8c10-bd2442327fa2 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:07.468070 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 669/1350] 10.222.0.93 () {54 vars in 921 bytes} [Mon Aug 1 15:04:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:07.473541 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85789fbb-c108-4226-ba46-576d4e7382c4 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:07.474668 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85789fbb-c108-4226-ba46-576d4e7382c4 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:07.475278 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85789fbb-c108-4226-ba46-576d4e7382c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:07.601885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-85789fbb-c108-4226-ba46-576d4e7382c4 None None] MFA Rules not processed for user `a07e3e25e6a8466fb237dcb836aecda2`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:07.605049 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-85789fbb-c108-4226-ba46-576d4e7382c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:07.641358 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:07.769630 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 682/1351] 10.222.0.93 () {58 vars in 1018 bytes} [Mon Aug 1 15:04:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:04:11.910285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b27a2e-1648-45ae-a0c6-746fc408e93d None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:11.910847 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b27a2e-1648-45ae-a0c6-746fc408e93d None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:11.910847 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b27a2e-1648-45ae-a0c6-746fc408e93d None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:11.912397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 670/1352] 10.222.0.93 () {54 vars in 921 bytes} [Mon Aug 1 15:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:11.917814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae135b96-13f3-4955-b84e-a65e3ea16c34 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:11.918281 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae135b96-13f3-4955-b84e-a65e3ea16c34 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:11.918461 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae135b96-13f3-4955-b84e-a65e3ea16c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:12.012092 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-ae135b96-13f3-4955-b84e-a65e3ea16c34 None None] MFA Rules not processed for user `a07e3e25e6a8466fb237dcb836aecda2`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:12.013418 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ae135b96-13f3-4955-b84e-a65e3ea16c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:12.095271 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:12.142184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 683/1353] 10.222.0.93 () {58 vars in 1018 bytes} [Mon Aug 1 15:04:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:04:13.619211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f53ce6-62aa-4dd8-b6a1-a5138141a1f8 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:13.619817 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f53ce6-62aa-4dd8-b6a1-a5138141a1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:13.619817 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f53ce6-62aa-4dd8-b6a1-a5138141a1f8 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:13.621708 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 671/1354] 10.222.0.93 () {54 vars in 921 bytes} [Mon Aug 1 15:04:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:13.626794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7061e78-4651-41f3-bf9d-abdf7f36058c None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:13.627263 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7061e78-4651-41f3-bf9d-abdf7f36058c None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:13.627385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7061e78-4651-41f3-bf9d-abdf7f36058c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:13.707187 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-e7061e78-4651-41f3-bf9d-abdf7f36058c None None] MFA Rules not processed for user `a07e3e25e6a8466fb237dcb836aecda2`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:13.708416 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e7061e78-4651-41f3-bf9d-abdf7f36058c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:13.818834 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 684/1355] 10.222.0.93 () {58 vars in 1018 bytes} [Mon Aug 1 15:04:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:04:17.481234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d2627-daa2-4e9c-a3bf-85d2e960aad0 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:17.481946 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d2627-daa2-4e9c-a3bf-85d2e960aad0 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:17.481946 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d2627-daa2-4e9c-a3bf-85d2e960aad0 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:17.483144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 672/1356] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:04:17.493209 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437e17b-3902-46a7-bf07-eb18dda2bae1 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:17.493770 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437e17b-3902-46a7-bf07-eb18dda2bae1 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:17.495198 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437e17b-3902-46a7-bf07-eb18dda2bae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:17.540810 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:17.599312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-c437e17b-3902-46a7-bf07-eb18dda2bae1 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:17.601302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-c437e17b-3902-46a7-bf07-eb18dda2bae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:17.777461 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 685/1357] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:04:17.840894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07af65a-42eb-49de-9563-51745e13b62a None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:17.841432 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07af65a-42eb-49de-9563-51745e13b62a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:17.841966 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07af65a-42eb-49de-9563-51745e13b62a None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:17.843869 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 673/1358] 10.222.0.93 () {54 vars in 962 bytes} [Mon Aug 1 15:04:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:17.852754 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7f52a-38b6-4b08-9c30-0bc07a36a993 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:17.853589 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7f52a-38b6-4b08-9c30-0bc07a36a993 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:17.855464 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7f52a-38b6-4b08-9c30-0bc07a36a993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:17.952106 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-c7b7f52a-38b6-4b08-9c30-0bc07a36a993 None None] MFA Rules not processed for user `3d9be74621d14c94b95ae94947e5abbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:17.953654 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-c7b7f52a-38b6-4b08-9c30-0bc07a36a993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:18.022591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:18.104796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 686/1359] 10.222.0.93 () {58 vars in 1059 bytes} [Mon Aug 1 15:04:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:04:18.111136 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07af65a-42eb-49de-9563-51745e13b62a None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:18.112839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a07af65a-42eb-49de-9563-51745e13b62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:18.288545 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07af65a-42eb-49de-9563-51745e13b62a None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:18.374950 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:18.423088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43454724-511a-4da7-ab5c-d60d029a0398 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:18.424245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43454724-511a-4da7-ab5c-d60d029a0398 service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:18.424740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43454724-511a-4da7-ab5c-d60d029a0398 service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:18.425192 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43454724-511a-4da7-ab5c-d60d029a0398 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:18.427105 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-43454724-511a-4da7-ab5c-d60d029a0398 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:18.534870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43454724-511a-4da7-ab5c-d60d029a0398 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:18.536552 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43454724-511a-4da7-ab5c-d60d029a0398 service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:18.555270 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 674/1360] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:04:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:04:20.876426 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623895e-db5e-4cb2-bc51-83e71df914a3 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:20.877031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623895e-db5e-4cb2-bc51-83e71df914a3 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:20.877405 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623895e-db5e-4cb2-bc51-83e71df914a3 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:20.879114 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 687/1361] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:04:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:20.886086 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c22ac-99be-40d3-9dac-36242709ddee None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:20.886636 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c22ac-99be-40d3-9dac-36242709ddee None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:20.886636 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c22ac-99be-40d3-9dac-36242709ddee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:21.001439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-ad1c22ac-99be-40d3-9dac-36242709ddee None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:21.003066 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ad1c22ac-99be-40d3-9dac-36242709ddee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:21.063271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:21.209082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 675/1362] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:04:21.249142 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623895e-db5e-4cb2-bc51-83e71df914a3 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:21.251133 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e623895e-db5e-4cb2-bc51-83e71df914a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:21.434078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623895e-db5e-4cb2-bc51-83e71df914a3 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:21.445489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:21.590095 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08b01c-939b-45a0-a065-508704a7a2f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:21.590799 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08b01c-939b-45a0-a065-508704a7a2f6 service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:21.591453 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08b01c-939b-45a0-a065-508704a7a2f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:21.591914 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08b01c-939b-45a0-a065-508704a7a2f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:21.595237 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-aa08b01c-939b-45a0-a065-508704a7a2f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:21.742159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa08b01c-939b-45a0-a065-508704a7a2f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:21.743975 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa08b01c-939b-45a0-a065-508704a7a2f6 service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:21.774560 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 688/1363] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:04:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:04:24.210069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5de9fa-1e11-47b8-9876-597c3ebc3d5d None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:24.210914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5de9fa-1e11-47b8-9876-597c3ebc3d5d None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:24.211183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5de9fa-1e11-47b8-9876-597c3ebc3d5d None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:24.212762 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 676/1364] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:04:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:24.221234 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8d9e1-1a85-4ce0-b899-30fad654e7fb None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:24.221975 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8d9e1-1a85-4ce0-b899-30fad654e7fb None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:24.221975 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8d9e1-1a85-4ce0-b899-30fad654e7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:24.250681 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:24.325460 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-37d8d9e1-1a85-4ce0-b899-30fad654e7fb None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:24.326906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-37d8d9e1-1a85-4ce0-b899-30fad654e7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:24.460612 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 689/1365] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:04:24.504721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c2538-3acb-498e-9c82-7c042df3cc0d None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:24.505329 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c2538-3acb-498e-9c82-7c042df3cc0d None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:24.505787 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c2538-3acb-498e-9c82-7c042df3cc0d None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:24.509010 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 677/1366] 10.222.0.93 () {54 vars in 962 bytes} [Mon Aug 1 15:04:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:24.517339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1c804-748b-466a-b1af-f42c098b746f None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:24.517666 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1c804-748b-466a-b1af-f42c098b746f None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:24.517908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1c804-748b-466a-b1af-f42c098b746f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:24.614209 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-97f1c804-748b-466a-b1af-f42c098b746f None None] MFA Rules not processed for user `3d9be74621d14c94b95ae94947e5abbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:24.615439 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-97f1c804-748b-466a-b1af-f42c098b746f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:24.664602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:24.783473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 690/1367] 10.222.0.93 () {58 vars in 1059 bytes} [Mon Aug 1 15:04:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:04:24.791359 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c2538-3acb-498e-9c82-7c042df3cc0d None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:24.792987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-074c2538-3acb-498e-9c82-7c042df3cc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:24.971152 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c2538-3acb-498e-9c82-7c042df3cc0d None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:24.997388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:25.113111 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7d89c-1ea7-46c3-95a6-9b56a459f456 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:25.113730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7d89c-1ea7-46c3-95a6-9b56a459f456 service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:25.114008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7d89c-1ea7-46c3-95a6-9b56a459f456 service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:25.114295 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7d89c-1ea7-46c3-95a6-9b56a459f456 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:25.116197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-48d7d89c-1ea7-46c3-95a6-9b56a459f456 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:25.247679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7d89c-1ea7-46c3-95a6-9b56a459f456 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:25.249327 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7d89c-1ea7-46c3-95a6-9b56a459f456 service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:25.270788 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 678/1368] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:04:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:04:26.138994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66869b-ed43-49ba-a99d-bb74244b3763 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:26.139572 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66869b-ed43-49ba-a99d-bb74244b3763 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:26.139670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66869b-ed43-49ba-a99d-bb74244b3763 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:26.143495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 691/1369] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:04:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:26.148728 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb248bc-f498-4518-ba86-97a9eb38f13c None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:26.149113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb248bc-f498-4518-ba86-97a9eb38f13c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:26.149300 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb248bc-f498-4518-ba86-97a9eb38f13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:26.191310 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:26.232291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-fbb248bc-f498-4518-ba86-97a9eb38f13c None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:26.233545 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-fbb248bc-f498-4518-ba86-97a9eb38f13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:26.362547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 679/1370] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:04:26.682959 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b576c1-a205-44ce-a2ea-9e420a5b0dff None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:26.683264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b576c1-a205-44ce-a2ea-9e420a5b0dff None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:26.683511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b576c1-a205-44ce-a2ea-9e420a5b0dff None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:26.685413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 692/1371] 10.222.0.93 () {54 vars in 964 bytes} [Mon Aug 1 15:04:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:26.691596 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974ac50-81de-49b1-94d4-0e50e1707686 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:26.691935 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974ac50-81de-49b1-94d4-0e50e1707686 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:26.692168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974ac50-81de-49b1-94d4-0e50e1707686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:26.790731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-5974ac50-81de-49b1-94d4-0e50e1707686 None None] MFA Rules not processed for user `67bf3011e2bd47a3b5cf5e0b1173f065`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:26.791857 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-5974ac50-81de-49b1-94d4-0e50e1707686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:26.847767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:26.891158 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 680/1372] 10.222.0.93 () {58 vars in 1061 bytes} [Mon Aug 1 15:04:26 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:04:26.907915 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b576c1-a205-44ce-a2ea-9e420a5b0dff None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:26.911256 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b4b576c1-a205-44ce-a2ea-9e420a5b0dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:27.039519 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b576c1-a205-44ce-a2ea-9e420a5b0dff None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:27.091047 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:27.136905 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917342b5-b090-4ede-85e4-66dbebfb18a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:27.139813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917342b5-b090-4ede-85e4-66dbebfb18a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:27.139813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917342b5-b090-4ede-85e4-66dbebfb18a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:27.139813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917342b5-b090-4ede-85e4-66dbebfb18a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:27.140641 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-917342b5-b090-4ede-85e4-66dbebfb18a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:27.281894 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917342b5-b090-4ede-85e4-66dbebfb18a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:27.283490 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917342b5-b090-4ede-85e4-66dbebfb18a0 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:27.317046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 693/1373] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:04:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:04:27.791424 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63016ff-98ba-4c1b-ae6a-d5192ebf6acf None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:27.791960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63016ff-98ba-4c1b-ae6a-d5192ebf6acf None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:27.791960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63016ff-98ba-4c1b-ae6a-d5192ebf6acf None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:27.793369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 681/1374] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:27.799486 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184820b-0330-4467-8862-42f54e7137da None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:27.799771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184820b-0330-4467-8862-42f54e7137da None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:27.800052 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184820b-0330-4467-8862-42f54e7137da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:27.880205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-d184820b-0330-4467-8862-42f54e7137da None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:27.881668 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-d184820b-0330-4467-8862-42f54e7137da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:27.907266 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:28.010673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 694/1375] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:04:28.058391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63016ff-98ba-4c1b-ae6a-d5192ebf6acf None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:28.059970 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c63016ff-98ba-4c1b-ae6a-d5192ebf6acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:28.235612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63016ff-98ba-4c1b-ae6a-d5192ebf6acf None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:28.298956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:28.371188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2c50e-3724-4285-8321-11df63512e16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:28.371800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2c50e-3724-4285-8321-11df63512e16 service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:28.372078 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2c50e-3724-4285-8321-11df63512e16 service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:28.372344 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2c50e-3724-4285-8321-11df63512e16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:28.374049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-adc2c50e-3724-4285-8321-11df63512e16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:28.472236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2c50e-3724-4285-8321-11df63512e16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:28.473914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2c50e-3724-4285-8321-11df63512e16 service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:28.495393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 682/1376] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:04:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:04:30.607720 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ed7e1-b155-4a5c-952f-33875831e76e None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:30.608221 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ed7e1-b155-4a5c-952f-33875831e76e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:30.608380 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ed7e1-b155-4a5c-952f-33875831e76e None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:30.609893 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 695/1377] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:30.615802 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daf32f-66f4-4cb9-a67a-ebd97365b35b None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:30.616237 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daf32f-66f4-4cb9-a67a-ebd97365b35b None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:30.616354 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daf32f-66f4-4cb9-a67a-ebd97365b35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:30.691138 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-16daf32f-66f4-4cb9-a67a-ebd97365b35b None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:30.692741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-16daf32f-66f4-4cb9-a67a-ebd97365b35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:30.702935 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:30.814345 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 683/1378] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:04:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:04:30.865429 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ed7e1-b155-4a5c-952f-33875831e76e None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:30.867226 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-117ed7e1-b155-4a5c-952f-33875831e76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:31.098901 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ed7e1-b155-4a5c-952f-33875831e76e None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:31.108314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:31.257093 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e313fe-4f3c-403a-af81-af94d6661c98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:31.257979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e313fe-4f3c-403a-af81-af94d6661c98 service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:31.258361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e313fe-4f3c-403a-af81-af94d6661c98 service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:31.258704 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e313fe-4f3c-403a-af81-af94d6661c98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:31.261305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-63e313fe-4f3c-403a-af81-af94d6661c98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:31.403580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e313fe-4f3c-403a-af81-af94d6661c98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:31.405159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e313fe-4f3c-403a-af81-af94d6661c98 service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:31.428564 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 696/1379] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:04:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:04:33.646012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640ac3d-0f10-402c-a98f-0976ea1bbd2f None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:33.646486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640ac3d-0f10-402c-a98f-0976ea1bbd2f None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:33.646611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640ac3d-0f10-402c-a98f-0976ea1bbd2f None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:33.647913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 684/1380] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:33.653974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0578d-f011-4be6-8294-474ff5ee3c81 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:33.654412 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0578d-f011-4be6-8294-474ff5ee3c81 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:33.654510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0578d-f011-4be6-8294-474ff5ee3c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:33.668006 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:33.737587 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-59a0578d-f011-4be6-8294-474ff5ee3c81 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:33.738807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-59a0578d-f011-4be6-8294-474ff5ee3c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:33.877574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 697/1381] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:04:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:04:33.925150 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640ac3d-0f10-402c-a98f-0976ea1bbd2f None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:33.926965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b640ac3d-0f10-402c-a98f-0976ea1bbd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:34.093793 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:34.105677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640ac3d-0f10-402c-a98f-0976ea1bbd2f None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:34.254585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b194d-b9df-405b-a1e3-ec424bca8cc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:34.255944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b194d-b9df-405b-a1e3-ec424bca8cc8 service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:34.256333 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b194d-b9df-405b-a1e3-ec424bca8cc8 service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:34.256735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b194d-b9df-405b-a1e3-ec424bca8cc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:34.258542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-772b194d-b9df-405b-a1e3-ec424bca8cc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:34.362651 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b194d-b9df-405b-a1e3-ec424bca8cc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:34.364375 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b194d-b9df-405b-a1e3-ec424bca8cc8 service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:34.376782 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:34.386605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 685/1382] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:04:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:04:36.614876 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14960931-97dc-40da-ac19-22e838c9528a None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:36.615411 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14960931-97dc-40da-ac19-22e838c9528a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:36.615411 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14960931-97dc-40da-ac19-22e838c9528a None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:36.617136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 698/1383] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:36.622788 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e6351-d192-496a-9134-3197ac2c9c1a None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:36.623324 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e6351-d192-496a-9134-3197ac2c9c1a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:36.623420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e6351-d192-496a-9134-3197ac2c9c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:36.825266 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-244e6351-d192-496a-9134-3197ac2c9c1a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:36.826987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-244e6351-d192-496a-9134-3197ac2c9c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:36.949232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 686/1384] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:04:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:04:36.997483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14960931-97dc-40da-ac19-22e838c9528a None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:36.999035 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-14960931-97dc-40da-ac19-22e838c9528a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:37.109177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:37.181219 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14960931-97dc-40da-ac19-22e838c9528a None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:37.354471 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0cf26d-f930-4c82-b4ed-9cab61973001 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:37.355160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cf26d-f930-4c82-b4ed-9cab61973001 service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:37.355437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cf26d-f930-4c82-b4ed-9cab61973001 service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:37.355721 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cf26d-f930-4c82-b4ed-9cab61973001 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:37.357559 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-8c0cf26d-f930-4c82-b4ed-9cab61973001 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:37.422418 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:37.475097 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0cf26d-f930-4c82-b4ed-9cab61973001 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:37.478108 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0cf26d-f930-4c82-b4ed-9cab61973001 service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:37.506407 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 699/1385] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:04:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:04:40.138905 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c92fe-cba6-4962-a0f1-4cf21a300796 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:40.139499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c92fe-cba6-4962-a0f1-4cf21a300796 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:40.139499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c92fe-cba6-4962-a0f1-4cf21a300796 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:40.140920 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 687/1386] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:04:40.146752 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f24461-3da2-4653-86b9-c59061357ff2 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:40.147264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f24461-3da2-4653-86b9-c59061357ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:40.147381 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f24461-3da2-4653-86b9-c59061357ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:40.226138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-d2f24461-3da2-4653-86b9-c59061357ff2 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:40.227336 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-d2f24461-3da2-4653-86b9-c59061357ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:40.345321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 700/1387] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:04:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:04:40.394046 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c92fe-cba6-4962-a0f1-4cf21a300796 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:40.395594 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-1c7c92fe-cba6-4962-a0f1-4cf21a300796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:40.477079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:40.588366 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c92fe-cba6-4962-a0f1-4cf21a300796 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:40.749942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cebd07e-bb9d-4e1e-8e83-5bb5e35aad66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:40.750506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebd07e-bb9d-4e1e-8e83-5bb5e35aad66 service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:40.751427 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebd07e-bb9d-4e1e-8e83-5bb5e35aad66 service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:40.751709 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebd07e-bb9d-4e1e-8e83-5bb5e35aad66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:40.753482 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-3cebd07e-bb9d-4e1e-8e83-5bb5e35aad66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:40.810585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:40.883207 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cebd07e-bb9d-4e1e-8e83-5bb5e35aad66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:40.885242 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cebd07e-bb9d-4e1e-8e83-5bb5e35aad66 service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:40.908643 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 688/1388] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:04:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:04:43.049087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50beb7-0452-4627-8a59-f4647bdf7fea None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:43.049781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50beb7-0452-4627-8a59-f4647bdf7fea None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:43.049781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50beb7-0452-4627-8a59-f4647bdf7fea None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:43.051040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 701/1389] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:43.058249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c39eb8-ef5d-496d-8415-71b5e3225440 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:43.058983 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c39eb8-ef5d-496d-8415-71b5e3225440 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:43.058983 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c39eb8-ef5d-496d-8415-71b5e3225440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:43.102903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a7250-baf7-4173-baff-7a3f165cb1cd None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:43.104095 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a7250-baf7-4173-baff-7a3f165cb1cd None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:43.104613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a7250-baf7-4173-baff-7a3f165cb1cd None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:43.107121 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 702/1390] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:04:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:43.117807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ceae-a023-4801-9c37-61ef417a0ecf None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:43.118281 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ceae-a023-4801-9c37-61ef417a0ecf None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:43.118765 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ceae-a023-4801-9c37-61ef417a0ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:43.138879 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-d7c39eb8-ef5d-496d-8415-71b5e3225440 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:43.140127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d7c39eb8-ef5d-496d-8415-71b5e3225440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:43.160995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:43.202198 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-f5b2ceae-a023-4801-9c37-61ef417a0ecf None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:43.204072 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f5b2ceae-a023-4801-9c37-61ef417a0ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:43.282881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 689/1391] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:04:43.326791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 703/1392] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:04:43.330160 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c39eb8-ef5d-496d-8415-71b5e3225440 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:43.331583 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d7c39eb8-ef5d-496d-8415-71b5e3225440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:43.346919 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:43.388828 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08500b96-46f0-4068-9349-0dc3f419cc5d None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:43.389175 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08500b96-46f0-4068-9349-0dc3f419cc5d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:43.389480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08500b96-46f0-4068-9349-0dc3f419cc5d None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:43.390927 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 704/1393] 10.222.0.93 () {54 vars in 966 bytes} [Mon Aug 1 15:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:43.396491 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf667f6-a629-41ee-8073-777501c2d670 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:43.396778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf667f6-a629-41ee-8073-777501c2d670 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:43.397051 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf667f6-a629-41ee-8073-777501c2d670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:43.471738 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c39eb8-ef5d-496d-8415-71b5e3225440 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:43.478963 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-edf667f6-a629-41ee-8073-777501c2d670 None None] MFA Rules not processed for user `4461a389da5848f5afd9854758707829`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:43.480648 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-edf667f6-a629-41ee-8073-777501c2d670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:43.535956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:43.565641 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 705/1394] 10.222.0.93 () {58 vars in 1063 bytes} [Mon Aug 1 15:04:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:04:43.573718 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf667f6-a629-41ee-8073-777501c2d670 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:43.581680 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf667f6-a629-41ee-8073-777501c2d670 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:43.598141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b4784-8065-4050-8a1c-fd1fde8c86a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:43.598755 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b4784-8065-4050-8a1c-fd1fde8c86a2 service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:43.599034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b4784-8065-4050-8a1c-fd1fde8c86a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:43.599304 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b4784-8065-4050-8a1c-fd1fde8c86a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:43.600973 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b38b4784-8065-4050-8a1c-fd1fde8c86a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:43.610201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:43.660396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8d709-c7ec-45f6-9d65-9d7b496f85ea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4461a389da5848f5afd9854758707829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:43.661042 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8d709-c7ec-45f6-9d65-9d7b496f85ea service glance] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:43.661322 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8d709-c7ec-45f6-9d65-9d7b496f85ea service glance] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:43.661592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8d709-c7ec-45f6-9d65-9d7b496f85ea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:43.663238 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-cfb8d709-c7ec-45f6-9d65-9d7b496f85ea service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:43.696188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b4784-8065-4050-8a1c-fd1fde8c86a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:43.698419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b4784-8065-4050-8a1c-fd1fde8c86a2 service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:43.718006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 690/1395] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:04:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:04:43.787346 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8d709-c7ec-45f6-9d65-9d7b496f85ea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:43.789124 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8d709-c7ec-45f6-9d65-9d7b496f85ea service glance] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:43.807639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:43.813684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 706/1396] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:04:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:04:44.585954 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72618a-7e39-44d9-8fe1-56050bc042d0 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:44.586476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72618a-7e39-44d9-8fe1-56050bc042d0 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:44.586476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72618a-7e39-44d9-8fe1-56050bc042d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:44.653618 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-6b72618a-7e39-44d9-8fe1-56050bc042d0 None None] MFA Rules not processed for user `4370f8f474c2446e989d0d8c5b75e87a`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:44.654825 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-6b72618a-7e39-44d9-8fe1-56050bc042d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:44.739703 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:44.744083 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 691/1397] 10.222.0.93 () {58 vars in 1009 bytes} [Mon Aug 1 15:04:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:04:44.792854 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f59d03-3a3a-45c7-b059-31fded019f9a None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:44.793529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f59d03-3a3a-45c7-b059-31fded019f9a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:44.793529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f59d03-3a3a-45c7-b059-31fded019f9a None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:44.794741 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 707/1398] 10.222.0.93 () {54 vars in 943 bytes} [Mon Aug 1 15:04:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:44.801139 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83203fcf-dc65-4d83-bc49-55d2a8d834f1 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:44.801386 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83203fcf-dc65-4d83-bc49-55d2a8d834f1 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:44.801610 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83203fcf-dc65-4d83-bc49-55d2a8d834f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:44.879884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-83203fcf-dc65-4d83-bc49-55d2a8d834f1 None None] MFA Rules not processed for user `4df37b02152c48509243b73ebbf49d8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:44.881145 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-83203fcf-dc65-4d83-bc49-55d2a8d834f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:45.004480 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 692/1399] 10.222.0.93 () {58 vars in 1040 bytes} [Mon Aug 1 15:04:44 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:04:45.014522 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f59d03-3a3a-45c7-b059-31fded019f9a None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:45.016038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-17f59d03-3a3a-45c7-b059-31fded019f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:45.157313 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f59d03-3a3a-45c7-b059-31fded019f9a None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:45.244542 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:45.265823 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4e1bb-30b1-4a40-9f87-70cfd2a17227 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4df37b02152c48509243b73ebbf49d8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:45.266347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4e1bb-30b1-4a40-9f87-70cfd2a17227 service swift] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:45.266626 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4e1bb-30b1-4a40-9f87-70cfd2a17227 service swift] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:45.266894 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4e1bb-30b1-4a40-9f87-70cfd2a17227 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:45.268595 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-60b4e1bb-30b1-4a40-9f87-70cfd2a17227 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:45.353078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b4e1bb-30b1-4a40-9f87-70cfd2a17227 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:45.354674 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b4e1bb-30b1-4a40-9f87-70cfd2a17227 service swift] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:45.363571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 708/1400] 10.222.0.93 () {60 vars in 1384 bytes} [Mon Aug 1 15:04:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:04:45.908896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7d6b3-b558-4314-85d4-303c69d68902 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:45.909465 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7d6b3-b558-4314-85d4-303c69d68902 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:45.909465 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7d6b3-b558-4314-85d4-303c69d68902 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:45.910693 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 693/1401] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:45.917087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69539b7f-65c4-4f99-bf55-45eef394e2a5 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:45.917666 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69539b7f-65c4-4f99-bf55-45eef394e2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:45.917666 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69539b7f-65c4-4f99-bf55-45eef394e2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:45.998503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-69539b7f-65c4-4f99-bf55-45eef394e2a5 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:46.001874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-69539b7f-65c4-4f99-bf55-45eef394e2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:46.126295 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:46.171639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 709/1402] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:04:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:04:46.216935 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7d6b3-b558-4314-85d4-303c69d68902 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:46.218487 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-24e7d6b3-b558-4314-85d4-303c69d68902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:46.299478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:46.396617 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7d6b3-b558-4314-85d4-303c69d68902 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:46.544147 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6962acb-d888-4201-8b2c-cfcda2171ed5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:46.544147 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6962acb-d888-4201-8b2c-cfcda2171ed5 service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:46.544147 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6962acb-d888-4201-8b2c-cfcda2171ed5 service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:46.544147 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6962acb-d888-4201-8b2c-cfcda2171ed5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:46.544950 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d6962acb-d888-4201-8b2c-cfcda2171ed5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:46.566334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:46.652248 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6962acb-d888-4201-8b2c-cfcda2171ed5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:46.654631 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6962acb-d888-4201-8b2c-cfcda2171ed5 service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:46.676686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 694/1403] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:04:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:04:48.798814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404b7ba-5932-49c9-8d1f-e67767dc6243 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:48.799344 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404b7ba-5932-49c9-8d1f-e67767dc6243 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:48.799344 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404b7ba-5932-49c9-8d1f-e67767dc6243 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:48.800959 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 710/1404] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:48.805937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c438b-ae64-44b8-8c68-13cdc919e297 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:48.806475 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c438b-ae64-44b8-8c68-13cdc919e297 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:48.806560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c438b-ae64-44b8-8c68-13cdc919e297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:48.885459 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-f84c438b-ae64-44b8-8c68-13cdc919e297 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:48.886693 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-f84c438b-ae64-44b8-8c68-13cdc919e297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:49.005895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:49.020703 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 695/1405] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:04:49.069952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2404b7ba-5932-49c9-8d1f-e67767dc6243 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:49.071435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-2404b7ba-5932-49c9-8d1f-e67767dc6243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:49.257089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2404b7ba-5932-49c9-8d1f-e67767dc6243 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:49.320560 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:49.388584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc508e1-64ae-4b1e-8de8-eccf9d014fad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:49.389225 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc508e1-64ae-4b1e-8de8-eccf9d014fad service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:49.389510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc508e1-64ae-4b1e-8de8-eccf9d014fad service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:49.389789 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc508e1-64ae-4b1e-8de8-eccf9d014fad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:49.391458 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0dc508e1-64ae-4b1e-8de8-eccf9d014fad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:49.493109 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc508e1-64ae-4b1e-8de8-eccf9d014fad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:49.495883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc508e1-64ae-4b1e-8de8-eccf9d014fad service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:49.522760 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 711/1406] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:04:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:04:51.623422 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba654b04-0e1e-4740-9999-cb5e2ecbe96b None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:51.623975 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba654b04-0e1e-4740-9999-cb5e2ecbe96b None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:51.623975 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba654b04-0e1e-4740-9999-cb5e2ecbe96b None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:51.625500 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 696/1407] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:04:51.630550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72c55d-f53c-406c-877d-d17d83b4f306 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:51.630994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72c55d-f53c-406c-877d-d17d83b4f306 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:51.631174 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72c55d-f53c-406c-877d-d17d83b4f306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:51.704965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-6f72c55d-f53c-406c-877d-d17d83b4f306 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:51.706200 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-6f72c55d-f53c-406c-877d-d17d83b4f306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:51.723432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:51.833959 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 712/1408] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:04:51.883347 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba654b04-0e1e-4740-9999-cb5e2ecbe96b None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:51.885177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ba654b04-0e1e-4740-9999-cb5e2ecbe96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:52.059331 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba654b04-0e1e-4740-9999-cb5e2ecbe96b None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:52.135471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:52.195498 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614743ca-3739-4f45-93f2-37346fbd6cb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:52.196129 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614743ca-3739-4f45-93f2-37346fbd6cb2 service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:52.196404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614743ca-3739-4f45-93f2-37346fbd6cb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:52.196743 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614743ca-3739-4f45-93f2-37346fbd6cb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:52.198397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-614743ca-3739-4f45-93f2-37346fbd6cb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:52.304996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614743ca-3739-4f45-93f2-37346fbd6cb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:52.308192 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614743ca-3739-4f45-93f2-37346fbd6cb2 service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:52.332857 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 697/1409] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:04:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:01.831556 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=38 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:02.330086 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=64 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:07.818957 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c704ecb-67a9-4360-9663-79474b5e96ae None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:07.819531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c704ecb-67a9-4360-9663-79474b5e96ae None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:07.819531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c704ecb-67a9-4360-9663-79474b5e96ae None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:07.820805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 713/1410] 10.222.0.93 () {54 vars in 918 bytes} [Mon Aug 1 15:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:07.826039 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f46e0-75cb-4544-ab50-958d7e5ea31a None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:07.826469 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f46e0-75cb-4544-ab50-958d7e5ea31a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:07.826586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f46e0-75cb-4544-ab50-958d7e5ea31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:07.904269 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-a16f46e0-75cb-4544-ab50-958d7e5ea31a None None] MFA Rules not processed for user `a07e3e25e6a8466fb237dcb836aecda2`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:07.905583 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a16f46e0-75cb-4544-ab50-958d7e5ea31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:08.010854 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 698/1411] 10.222.0.93 () {58 vars in 1015 bytes} [Mon Aug 1 15:05:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:05:08.025311 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658b0cb-b6d1-44ad-8142-72cbdec7dd17 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:08.025594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658b0cb-b6d1-44ad-8142-72cbdec7dd17 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:08.025867 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658b0cb-b6d1-44ad-8142-72cbdec7dd17 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:08.027224 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 714/1412] 10.222.0.93 () {54 vars in 918 bytes} [Mon Aug 1 15:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:08.031956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964efc8-9f5a-4f28-90e8-cffc0f86c236 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:08.032245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964efc8-9f5a-4f28-90e8-cffc0f86c236 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:08.032555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964efc8-9f5a-4f28-90e8-cffc0f86c236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:08.106541 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:08.109279 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-a964efc8-9f5a-4f28-90e8-cffc0f86c236 None None] MFA Rules not processed for user `3d9be74621d14c94b95ae94947e5abbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:08.111133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a964efc8-9f5a-4f28-90e8-cffc0f86c236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:08.241894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 699/1413] 10.222.0.93 () {58 vars in 1015 bytes} [Mon Aug 1 15:05:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:10.773535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77efb1-502b-4aff-a3ed-3f3c26571f27 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:10.774085 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77efb1-502b-4aff-a3ed-3f3c26571f27 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:10.774085 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77efb1-502b-4aff-a3ed-3f3c26571f27 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:10.775445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:10.781431 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42725f5f-df90-40d1-ab2d-af93384e578a None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:10.781984 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42725f5f-df90-40d1-ab2d-af93384e578a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:10.781984 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42725f5f-df90-40d1-ab2d-af93384e578a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:10.855372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-42725f5f-df90-40d1-ab2d-af93384e578a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:10.856615 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-42725f5f-df90-40d1-ab2d-af93384e578a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:10.914110 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:10.958058 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:11.002790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77efb1-502b-4aff-a3ed-3f3c26571f27 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:11.004410 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5e77efb1-502b-4aff-a3ed-3f3c26571f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:11.154453 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77efb1-502b-4aff-a3ed-3f3c26571f27 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:11.256261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:11.306839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33a038-c861-4410-a699-0abb51174ee8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:11.307561 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33a038-c861-4410-a699-0abb51174ee8 service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:11.307842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33a038-c861-4410-a699-0abb51174ee8 service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:11.308117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33a038-c861-4410-a699-0abb51174ee8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:11.310291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-8d33a038-c861-4410-a699-0abb51174ee8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:11.450395 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33a038-c861-4410-a699-0abb51174ee8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:11.452001 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33a038-c861-4410-a699-0abb51174ee8 service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:11.475310 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 716/1416] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:05:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:12.420351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd9cfd-d403-4ea0-b922-83e33508c3c0 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:12.421083 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd9cfd-d403-4ea0-b922-83e33508c3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:12.421272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd9cfd-d403-4ea0-b922-83e33508c3c0 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:12.422791 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:05:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:12.428528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbf638-c65d-48ab-b39c-e56a74e535a1 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:12.429096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbf638-c65d-48ab-b39c-e56a74e535a1 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:12.429096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbf638-c65d-48ab-b39c-e56a74e535a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:12.518514 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-23dbf638-c65d-48ab-b39c-e56a74e535a1 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:12.519760 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-23dbf638-c65d-48ab-b39c-e56a74e535a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:12.556331 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:12.637536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:12.788300 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd9cfd-d403-4ea0-b922-83e33508c3c0 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:12.789784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-58bd9cfd-d403-4ea0-b922-83e33508c3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:12.944595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd9cfd-d403-4ea0-b922-83e33508c3c0 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:12.988000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:13.074994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5d420-f253-40f9-87ca-458a70ab516e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:13.075634 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5d420-f253-40f9-87ca-458a70ab516e service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:13.075907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5d420-f253-40f9-87ca-458a70ab516e service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:13.076171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5d420-f253-40f9-87ca-458a70ab516e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:13.077882 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-43d5d420-f253-40f9-87ca-458a70ab516e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:13.175953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5d420-f253-40f9-87ca-458a70ab516e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:13.177631 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5d420-f253-40f9-87ca-458a70ab516e service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:13.199321 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 702/1419] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:05:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:14.120329 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44250b57-36d0-4e2b-8d0e-3b9b76ad24cb None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:14.120898 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44250b57-36d0-4e2b-8d0e-3b9b76ad24cb None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:14.120898 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44250b57-36d0-4e2b-8d0e-3b9b76ad24cb None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:14.122258 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:14.127519 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178b36e-a383-4412-a915-49b241677d48 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:14.128143 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178b36e-a383-4412-a915-49b241677d48 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:14.128143 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178b36e-a383-4412-a915-49b241677d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:14.185752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:14.210008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-4178b36e-a383-4412-a915-49b241677d48 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:14.211252 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-4178b36e-a383-4412-a915-49b241677d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:14.319882 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:05:14.474528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44250b57-36d0-4e2b-8d0e-3b9b76ad24cb None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:14.476075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-44250b57-36d0-4e2b-8d0e-3b9b76ad24cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:14.639605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44250b57-36d0-4e2b-8d0e-3b9b76ad24cb None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:14.652672 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:14.768274 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f13335-78bc-4e09-b7eb-d9e8b5c13383 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:14.769031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f13335-78bc-4e09-b7eb-d9e8b5c13383 service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:14.769336 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f13335-78bc-4e09-b7eb-d9e8b5c13383 service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:14.769660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f13335-78bc-4e09-b7eb-d9e8b5c13383 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:14.771420 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-43f13335-78bc-4e09-b7eb-d9e8b5c13383 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:14.884665 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f13335-78bc-4e09-b7eb-d9e8b5c13383 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:14.886688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f13335-78bc-4e09-b7eb-d9e8b5c13383 service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:14.905838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 719/1422] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:05:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:15.785144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b416f4-4a39-4782-ae49-120838a7111a None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:15.785660 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b416f4-4a39-4782-ae49-120838a7111a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:15.785660 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b416f4-4a39-4782-ae49-120838a7111a None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:15.786994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:15.792496 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e775b7b-43d6-4f41-be6c-3df6b65e6987 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:15.793155 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e775b7b-43d6-4f41-be6c-3df6b65e6987 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:15.793155 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e775b7b-43d6-4f41-be6c-3df6b65e6987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:15.814362 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:15.878601 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-6e775b7b-43d6-4f41-be6c-3df6b65e6987 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:15.880056 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-6e775b7b-43d6-4f41-be6c-3df6b65e6987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:15.979615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:16.133959 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b416f4-4a39-4782-ae49-120838a7111a None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:16.135581 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-70b416f4-4a39-4782-ae49-120838a7111a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:16.251712 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:16.288136 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b416f4-4a39-4782-ae49-120838a7111a None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:16.406691 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3cabe4-700b-49d9-bb4e-2e306567d960 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:16.407222 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cabe4-700b-49d9-bb4e-2e306567d960 service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:16.407581 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cabe4-700b-49d9-bb4e-2e306567d960 service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:16.407776 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cabe4-700b-49d9-bb4e-2e306567d960 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:16.409573 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-8f3cabe4-700b-49d9-bb4e-2e306567d960 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:16.497505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:16.514122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3cabe4-700b-49d9-bb4e-2e306567d960 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:16.515760 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3cabe4-700b-49d9-bb4e-2e306567d960 service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:16.536126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 705/1425] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:05:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:17.419871 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5feabf-91b0-4965-a9cd-34d2114e7fe8 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:17.420629 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5feabf-91b0-4965-a9cd-34d2114e7fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:17.420629 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5feabf-91b0-4965-a9cd-34d2114e7fe8 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:17.421890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:17.427209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcff23-c971-45b7-b1a2-bee0a07019b9 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:17.427701 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcff23-c971-45b7-b1a2-bee0a07019b9 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:17.427806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcff23-c971-45b7-b1a2-bee0a07019b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:17.508748 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-83dcff23-c971-45b7-b1a2-bee0a07019b9 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:17.510591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-83dcff23-c971-45b7-b1a2-bee0a07019b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:17.616364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:17.773606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5feabf-91b0-4965-a9cd-34d2114e7fe8 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:17.776590 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ea5feabf-91b0-4965-a9cd-34d2114e7fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:17.864539 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:17.929162 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5feabf-91b0-4965-a9cd-34d2114e7fe8 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:18.043169 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f8750-47a6-467d-b623-afcd8e091a6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:18.043650 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f8750-47a6-467d-b623-afcd8e091a6d service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:18.043930 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f8750-47a6-467d-b623-afcd8e091a6d service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:18.044201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f8750-47a6-467d-b623-afcd8e091a6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:18.045879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-585f8750-47a6-467d-b623-afcd8e091a6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:18.095094 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:18.137881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f8750-47a6-467d-b623-afcd8e091a6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:18.139536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f8750-47a6-467d-b623-afcd8e091a6d service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:18.157931 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 722/1428] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:05:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:18.977974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09198ee5-b7e0-4384-b07c-9ac39e9a0e77 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:18.978529 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09198ee5-b7e0-4384-b07c-9ac39e9a0e77 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:18.978529 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09198ee5-b7e0-4384-b07c-9ac39e9a0e77 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:18.979888 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:18.987615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0673f-23b3-43fb-99bd-5976abe6992a None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:18.988152 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0673f-23b3-43fb-99bd-5976abe6992a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:18.988152 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0673f-23b3-43fb-99bd-5976abe6992a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:19.082007 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-22d0673f-23b3-43fb-99bd-5976abe6992a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:19.083415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-22d0673f-23b3-43fb-99bd-5976abe6992a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:19.192289 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:19.348308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09198ee5-b7e0-4384-b07c-9ac39e9a0e77 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:19.349893 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-09198ee5-b7e0-4384-b07c-9ac39e9a0e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:19.403780 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:19.507027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09198ee5-b7e0-4384-b07c-9ac39e9a0e77 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:19.623788 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5fbf7-22d5-4b71-9640-a9c3dd0456cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:19.624309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5fbf7-22d5-4b71-9640-a9c3dd0456cb service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:19.624614 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5fbf7-22d5-4b71-9640-a9c3dd0456cb service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:19.624884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5fbf7-22d5-4b71-9640-a9c3dd0456cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:19.626507 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-83d5fbf7-22d5-4b71-9640-a9c3dd0456cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:19.633654 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:19.772265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5fbf7-22d5-4b71-9640-a9c3dd0456cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:19.775915 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5fbf7-22d5-4b71-9640-a9c3dd0456cb service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:19.801137 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 708/1431] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:20.625973 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be399d0-3d19-4b18-918c-7ac5802d15ea None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:20.626570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be399d0-3d19-4b18-918c-7ac5802d15ea None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:20.626570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be399d0-3d19-4b18-918c-7ac5802d15ea None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:20.627926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:20.635254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113ad0a-fadd-4fe8-b329-a60a2eb0470a None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:20.635881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113ad0a-fadd-4fe8-b329-a60a2eb0470a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:20.636241 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113ad0a-fadd-4fe8-b329-a60a2eb0470a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:20.724888 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-7113ad0a-fadd-4fe8-b329-a60a2eb0470a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:20.726136 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-7113ad0a-fadd-4fe8-b329-a60a2eb0470a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:20.804942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:20.838447 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:20.992387 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be399d0-3d19-4b18-918c-7ac5802d15ea None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:20.994334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4be399d0-3d19-4b18-918c-7ac5802d15ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:21.013579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:21.185934 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be399d0-3d19-4b18-918c-7ac5802d15ea None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:21.318084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:21.351066 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e8fca-bdad-4305-a638-df05fed8ff4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:21.351756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e8fca-bdad-4305-a638-df05fed8ff4d service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:21.352091 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e8fca-bdad-4305-a638-df05fed8ff4d service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:21.352536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e8fca-bdad-4305-a638-df05fed8ff4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:21.354306 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-d97e8fca-bdad-4305-a638-df05fed8ff4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:21.482646 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e8fca-bdad-4305-a638-df05fed8ff4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:21.484190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e8fca-bdad-4305-a638-df05fed8ff4d service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:21.504800 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 725/1434] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:05:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:05:22.343491 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068c654-0230-4d7d-8bb3-24e44fa09b5d None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:22.344086 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068c654-0230-4d7d-8bb3-24e44fa09b5d None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:22.344086 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068c654-0230-4d7d-8bb3-24e44fa09b5d None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:22.345490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:05:22.350995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7e3d1-a109-4b51-b549-6b4a11b8ce30 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:22.351523 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7e3d1-a109-4b51-b549-6b4a11b8ce30 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:22.351655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7e3d1-a109-4b51-b549-6b4a11b8ce30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:22.435207 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-0ed7e3d1-a109-4b51-b549-6b4a11b8ce30 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:22.436420 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0ed7e3d1-a109-4b51-b549-6b4a11b8ce30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:22.489120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:22.556445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:22.711123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068c654-0230-4d7d-8bb3-24e44fa09b5d None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:22.712776 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-4068c654-0230-4d7d-8bb3-24e44fa09b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:22.889909 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068c654-0230-4d7d-8bb3-24e44fa09b5d None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:22.955806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:23.024616 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1797c-4256-48fb-86d7-9963d92db272 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:23.025275 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1797c-4256-48fb-86d7-9963d92db272 service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:23.025704 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1797c-4256-48fb-86d7-9963d92db272 service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:23.026090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1797c-4256-48fb-86d7-9963d92db272 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:23.027970 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-6ba1797c-4256-48fb-86d7-9963d92db272 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:23.145969 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1797c-4256-48fb-86d7-9963d92db272 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:23.147624 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1797c-4256-48fb-86d7-9963d92db272 service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:23.170435 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 711/1437] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:05:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:24.008868 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb9bc6-8732-4c87-a0d8-4bceb17deb97 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:24.009492 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb9bc6-8732-4c87-a0d8-4bceb17deb97 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:24.009492 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb9bc6-8732-4c87-a0d8-4bceb17deb97 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:24.010810 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:24.016330 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e617cf-c14c-4d52-8c5f-e4408b1225dc None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:24.016849 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e617cf-c14c-4d52-8c5f-e4408b1225dc None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:24.016945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e617cf-c14c-4d52-8c5f-e4408b1225dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:24.100735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-06e617cf-c14c-4d52-8c5f-e4408b1225dc None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:24.101971 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-06e617cf-c14c-4d52-8c5f-e4408b1225dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:24.104719 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:24.205392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:05:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:24.360598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb9bc6-8732-4c87-a0d8-4bceb17deb97 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:24.362687 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-deeb9bc6-8732-4c87-a0d8-4bceb17deb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:24.530582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb9bc6-8732-4c87-a0d8-4bceb17deb97 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:24.561441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:24.663181 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f691a-02c7-4a20-bb50-889404ddfe80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:24.663711 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f691a-02c7-4a20-bb50-889404ddfe80 service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:24.663998 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f691a-02c7-4a20-bb50-889404ddfe80 service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:24.664268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f691a-02c7-4a20-bb50-889404ddfe80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:24.666370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-690f691a-02c7-4a20-bb50-889404ddfe80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:24.766056 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f691a-02c7-4a20-bb50-889404ddfe80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:24.767703 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f691a-02c7-4a20-bb50-889404ddfe80 service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:24.788068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 728/1440] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:05:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:26.758141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a873f-2ace-4809-9da6-c459b9ef8956 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:26.758901 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a873f-2ace-4809-9da6-c459b9ef8956 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:26.758901 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a873f-2ace-4809-9da6-c459b9ef8956 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:26.760589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:26.766943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b4fc7-0927-4477-927a-3354828bb2b1 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:26.767788 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b4fc7-0927-4477-927a-3354828bb2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:26.767788 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b4fc7-0927-4477-927a-3354828bb2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:26.815269 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:26.851473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-b59b4fc7-0927-4477-927a-3354828bb2b1 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:26.853357 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b59b4fc7-0927-4477-927a-3354828bb2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:27.011448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:27.058057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a873f-2ace-4809-9da6-c459b9ef8956 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:27.060369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-289a873f-2ace-4809-9da6-c459b9ef8956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:27.192100 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a873f-2ace-4809-9da6-c459b9ef8956 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:27.228057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:27.277162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11a757-11e2-4b37-8ce3-a200f2e52c63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:27.277689 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a757-11e2-4b37-8ce3-a200f2e52c63 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:27.277971 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a757-11e2-4b37-8ce3-a200f2e52c63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:27.278250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a757-11e2-4b37-8ce3-a200f2e52c63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:27.279929 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-3f11a757-11e2-4b37-8ce3-a200f2e52c63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:27.406422 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11a757-11e2-4b37-8ce3-a200f2e52c63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:27.407944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11a757-11e2-4b37-8ce3-a200f2e52c63 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:27.429123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 714/1443] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:05:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:29.749542 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb04c9-c34b-4e5a-af55-dd31af310e31 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:29.750170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb04c9-c34b-4e5a-af55-dd31af310e31 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:29.750170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb04c9-c34b-4e5a-af55-dd31af310e31 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:29.751473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:29.757389 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6c807-78bb-46aa-8570-f53abbefac0e None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:29.757947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6c807-78bb-46aa-8570-f53abbefac0e None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:29.757947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6c807-78bb-46aa-8570-f53abbefac0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:29.859448 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-8ea6c807-78bb-46aa-8570-f53abbefac0e None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:29.860999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-8ea6c807-78bb-46aa-8570-f53abbefac0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:29.886302 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:30.000597 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:30.040162 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb04c9-c34b-4e5a-af55-dd31af310e31 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:30.041929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-7adb04c9-c34b-4e5a-af55-dd31af310e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:30.172361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb04c9-c34b-4e5a-af55-dd31af310e31 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:30.187031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:30.256527 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cafa3e7-d671-44c0-aa40-b5280b260d1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:30.257156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafa3e7-d671-44c0-aa40-b5280b260d1f service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:30.257433 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafa3e7-d671-44c0-aa40-b5280b260d1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:30.257787 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafa3e7-d671-44c0-aa40-b5280b260d1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:30.259940 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-2cafa3e7-d671-44c0-aa40-b5280b260d1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:30.382682 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cafa3e7-d671-44c0-aa40-b5280b260d1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:30.384946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cafa3e7-d671-44c0-aa40-b5280b260d1f service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:30.415302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 731/1446] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:05:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:32.843283 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42973239-65bf-4665-945c-d81443a1c107 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:32.843839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42973239-65bf-4665-945c-d81443a1c107 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:32.843839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42973239-65bf-4665-945c-d81443a1c107 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:32.845217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:32.850868 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754a8aa-53f5-4448-a7e1-4dee504dfc8b None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:32.851388 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754a8aa-53f5-4448-a7e1-4dee504dfc8b None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:32.851508 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754a8aa-53f5-4448-a7e1-4dee504dfc8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:32.904745 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:32.926007 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-1754a8aa-53f5-4448-a7e1-4dee504dfc8b None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:32.927823 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-1754a8aa-53f5-4448-a7e1-4dee504dfc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:33.037032 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:33.078757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42973239-65bf-4665-945c-d81443a1c107 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:33.080293 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-42973239-65bf-4665-945c-d81443a1c107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:33.202203 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42973239-65bf-4665-945c-d81443a1c107 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:33.266806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:33.307225 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf047a6-c25d-42ee-be19-f133df58968d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:33.307890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf047a6-c25d-42ee-be19-f133df58968d service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:33.308173 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf047a6-c25d-42ee-be19-f133df58968d service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:33.308450 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf047a6-c25d-42ee-be19-f133df58968d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:33.310205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-7bf047a6-c25d-42ee-be19-f133df58968d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:33.428358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf047a6-c25d-42ee-be19-f133df58968d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:33.429836 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf047a6-c25d-42ee-be19-f133df58968d service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:33.450884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 717/1449] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:05:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:35.638112 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a197855-be8e-4599-b7a2-3ed49af5c40c None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:35.638681 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a197855-be8e-4599-b7a2-3ed49af5c40c None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:35.638681 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a197855-be8e-4599-b7a2-3ed49af5c40c None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:35.640066 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:35.645483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c26580-a0c8-4219-bdb8-529f626e2574 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:35.645935 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c26580-a0c8-4219-bdb8-529f626e2574 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:35.646064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c26580-a0c8-4219-bdb8-529f626e2574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:35.715542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-49c26580-a0c8-4219-bdb8-529f626e2574 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:35.716790 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-49c26580-a0c8-4219-bdb8-529f626e2574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:35.738516 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:35.816340 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:35.856916 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a197855-be8e-4599-b7a2-3ed49af5c40c None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:35.858447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-8a197855-be8e-4599-b7a2-3ed49af5c40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:35.968552 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a197855-be8e-4599-b7a2-3ed49af5c40c None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:35.989872 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:36.046355 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568a2b0-137b-4e2a-8933-34ab8388e8a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:36.046976 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568a2b0-137b-4e2a-8933-34ab8388e8a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:36.047260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568a2b0-137b-4e2a-8933-34ab8388e8a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:36.047540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568a2b0-137b-4e2a-8933-34ab8388e8a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:36.049237 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e568a2b0-137b-4e2a-8933-34ab8388e8a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:36.173501 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568a2b0-137b-4e2a-8933-34ab8388e8a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:36.174973 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568a2b0-137b-4e2a-8933-34ab8388e8a3 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:36.198521 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 734/1452] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:05:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:38.293358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80143db4-38c3-451d-bce9-279ada4779c6 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:38.293873 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80143db4-38c3-451d-bce9-279ada4779c6 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:38.293873 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80143db4-38c3-451d-bce9-279ada4779c6 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:38.295214 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:38.300489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fda020-66e6-4a90-a7be-791873c9afcf None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:38.300940 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fda020-66e6-4a90-a7be-791873c9afcf None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:38.301046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fda020-66e6-4a90-a7be-791873c9afcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:38.364607 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:38.368246 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-b5fda020-66e6-4a90-a7be-791873c9afcf None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:38.370130 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b5fda020-66e6-4a90-a7be-791873c9afcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:38.472233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:38.511830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80143db4-38c3-451d-bce9-279ada4779c6 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:38.513392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-80143db4-38c3-451d-bce9-279ada4779c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:38.623026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80143db4-38c3-451d-bce9-279ada4779c6 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:38.675085 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:38.701058 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac2dd6-723e-416c-91cd-0e7dcedeaef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:38.701595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2dd6-723e-416c-91cd-0e7dcedeaef6 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:38.701873 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2dd6-723e-416c-91cd-0e7dcedeaef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:38.702138 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2dd6-723e-416c-91cd-0e7dcedeaef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:38.703786 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-7bac2dd6-723e-416c-91cd-0e7dcedeaef6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:38.821683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac2dd6-723e-416c-91cd-0e7dcedeaef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:38.823130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac2dd6-723e-416c-91cd-0e7dcedeaef6 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:38.842210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 720/1455] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:05:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:40.961084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec2df3-83b0-4ac8-97ef-6f9b57593542 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:40.961672 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec2df3-83b0-4ac8-97ef-6f9b57593542 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:40.961672 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec2df3-83b0-4ac8-97ef-6f9b57593542 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:40.963005 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:40.968601 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64209e-d5ca-43b7-b1ca-2b7918d61057 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:40.969103 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64209e-d5ca-43b7-b1ca-2b7918d61057 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:40.969201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64209e-d5ca-43b7-b1ca-2b7918d61057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:41.037896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-bf64209e-d5ca-43b7-b1ca-2b7918d61057 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:41.039134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-bf64209e-d5ca-43b7-b1ca-2b7918d61057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:41.070412 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:41.141294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:41.180756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec2df3-83b0-4ac8-97ef-6f9b57593542 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:41.182361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3bec2df3-83b0-4ac8-97ef-6f9b57593542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:41.309537 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec2df3-83b0-4ac8-97ef-6f9b57593542 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:41.343129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:41.393981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d8ec9-da66-4c8c-9851-1e936577de83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:41.394595 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d8ec9-da66-4c8c-9851-1e936577de83 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:41.394912 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d8ec9-da66-4c8c-9851-1e936577de83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:41.395154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d8ec9-da66-4c8c-9851-1e936577de83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:41.396885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-559d8ec9-da66-4c8c-9851-1e936577de83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:41.514936 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d8ec9-da66-4c8c-9851-1e936577de83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:41.516358 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d8ec9-da66-4c8c-9851-1e936577de83 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:41.535383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 737/1458] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:05:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:43.680337 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac8ed2-de3e-478d-aea6-58d6d1aa899a None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:43.681172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac8ed2-de3e-478d-aea6-58d6d1aa899a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:43.681172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac8ed2-de3e-478d-aea6-58d6d1aa899a None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:43.682820 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:05:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:43.688819 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d4349-dbd6-4efb-8d77-55f52764af9b None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:43.689368 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d4349-dbd6-4efb-8d77-55f52764af9b None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:43.689368 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d4349-dbd6-4efb-8d77-55f52764af9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:43.773274 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-1d2d4349-dbd6-4efb-8d77-55f52764af9b None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:43.775689 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-1d2d4349-dbd6-4efb-8d77-55f52764af9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:43.787530 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:43.893282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:43.933453 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac8ed2-de3e-478d-aea6-58d6d1aa899a None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:43.934926 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c8ac8ed2-de3e-478d-aea6-58d6d1aa899a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:44.065973 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac8ed2-de3e-478d-aea6-58d6d1aa899a None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:44.131701 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:44.162016 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f7af5-2d95-43be-81d3-25bf5e363238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:44.162645 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f7af5-2d95-43be-81d3-25bf5e363238 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:44.162934 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f7af5-2d95-43be-81d3-25bf5e363238 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:44.163209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f7af5-2d95-43be-81d3-25bf5e363238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:44.165550 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-132f7af5-2d95-43be-81d3-25bf5e363238 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:44.342062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f7af5-2d95-43be-81d3-25bf5e363238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:44.344951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f7af5-2d95-43be-81d3-25bf5e363238 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:44.383333 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 723/1461] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:05:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:46.599370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa1260-8448-47d0-8158-54dff0d51467 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:46.599932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa1260-8448-47d0-8158-54dff0d51467 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:46.599932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa1260-8448-47d0-8158-54dff0d51467 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:46.601399 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:46.607533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79d818-a14a-4289-8ba6-5a1172d9a079 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:46.608047 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79d818-a14a-4289-8ba6-5a1172d9a079 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:46.608145 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79d818-a14a-4289-8ba6-5a1172d9a079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:46.681161 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-ba79d818-a14a-4289-8ba6-5a1172d9a079 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:46.682372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ba79d818-a14a-4289-8ba6-5a1172d9a079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:46.723061 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:46.792235 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:46.832609 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa1260-8448-47d0-8158-54dff0d51467 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:46.834268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-80aa1260-8448-47d0-8158-54dff0d51467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:46.973303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa1260-8448-47d0-8158-54dff0d51467 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:47.024801 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:47.072335 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc29c3-bb7d-4f53-beb4-c7a705bb3f63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:47.072926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc29c3-bb7d-4f53-beb4-c7a705bb3f63 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:47.073203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc29c3-bb7d-4f53-beb4-c7a705bb3f63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:47.073481 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc29c3-bb7d-4f53-beb4-c7a705bb3f63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:47.075319 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f5cc29c3-bb7d-4f53-beb4-c7a705bb3f63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:47.220878 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc29c3-bb7d-4f53-beb4-c7a705bb3f63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:47.222347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc29c3-bb7d-4f53-beb4-c7a705bb3f63 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:47.244705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 740/1464] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:05:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:05:49.347249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d5398-b193-48e1-a0a9-8723f9d5b4d6 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:49.347858 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d5398-b193-48e1-a0a9-8723f9d5b4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:49.347858 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d5398-b193-48e1-a0a9-8723f9d5b4d6 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:49.349159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:49.355215 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286fb2d-201b-4ac8-bdb5-1a5a8130d63e None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:49.355719 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286fb2d-201b-4ac8-bdb5-1a5a8130d63e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:49.355830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286fb2d-201b-4ac8-bdb5-1a5a8130d63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:49.426712 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-f286fb2d-201b-4ac8-bdb5-1a5a8130d63e None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:49.427964 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f286fb2d-201b-4ac8-bdb5-1a5a8130d63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:49.447937 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:49.538434 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:49.576350 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d5398-b193-48e1-a0a9-8723f9d5b4d6 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:49.578041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-dd0d5398-b193-48e1-a0a9-8723f9d5b4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:49.695179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d5398-b193-48e1-a0a9-8723f9d5b4d6 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:49.785651 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:49.799792 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654d1aa-9ea6-4fff-8f7c-97cd8b0a4a1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:49.800876 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654d1aa-9ea6-4fff-8f7c-97cd8b0a4a1d service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:49.801372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654d1aa-9ea6-4fff-8f7c-97cd8b0a4a1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:49.801860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654d1aa-9ea6-4fff-8f7c-97cd8b0a4a1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:49.805106 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-6654d1aa-9ea6-4fff-8f7c-97cd8b0a4a1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:49.935006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654d1aa-9ea6-4fff-8f7c-97cd8b0a4a1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:49.936452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654d1aa-9ea6-4fff-8f7c-97cd8b0a4a1d service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:49.955810 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 726/1467] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:05:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:05:52.062261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69925b-6713-49f4-b162-1c821c29c5ea None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:52.062869 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69925b-6713-49f4-b162-1c821c29c5ea None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:52.062869 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69925b-6713-49f4-b162-1c821c29c5ea None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:52.064362 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:05:52.070513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac487b-195a-45a5-b118-909736776187 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:52.071061 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac487b-195a-45a5-b118-909736776187 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:52.071061 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac487b-195a-45a5-b118-909736776187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:52.156984 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-6eac487b-195a-45a5-b118-909736776187 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:52.158496 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-6eac487b-195a-45a5-b118-909736776187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:52.221664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:52.274812 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:52.315729 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69925b-6713-49f4-b162-1c821c29c5ea None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:52.319160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3a69925b-6713-49f4-b162-1c821c29c5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:52.492406 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69925b-6713-49f4-b162-1c821c29c5ea None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:52.545250 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:52.574454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a7bd8-4cd8-4154-b5d0-62ae17f51804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:52.575112 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a7bd8-4cd8-4154-b5d0-62ae17f51804 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:52.575363 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a7bd8-4cd8-4154-b5d0-62ae17f51804 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:52.575631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a7bd8-4cd8-4154-b5d0-62ae17f51804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:52.577368 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-de0a7bd8-4cd8-4154-b5d0-62ae17f51804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:52.706264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a7bd8-4cd8-4154-b5d0-62ae17f51804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:52.707709 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a7bd8-4cd8-4154-b5d0-62ae17f51804 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:52.728077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 743/1470] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:05:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:02.272946 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=22 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:02.725120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=66 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:12.619693 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab8189-a290-4c48-a4bf-b86e5f3e4eb1 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:12.620271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab8189-a290-4c48-a4bf-b86e5f3e4eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:12.620271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab8189-a290-4c48-a4bf-b86e5f3e4eb1 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:12.621632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 728/1471] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:06:12.627094 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451b0d6-cbde-46e8-8a95-7684ffe19e52 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:12.627559 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451b0d6-cbde-46e8-8a95-7684ffe19e52 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:12.627664 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451b0d6-cbde-46e8-8a95-7684ffe19e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:12.698047 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-a451b0d6-cbde-46e8-8a95-7684ffe19e52 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:12.699337 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a451b0d6-cbde-46e8-8a95-7684ffe19e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:12.799290 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 744/1472] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:12.888297 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad5add-d4c7-477a-a06a-b48436f66dd6 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:12.888605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad5add-d4c7-477a-a06a-b48436f66dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:12.888896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad5add-d4c7-477a-a06a-b48436f66dd6 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:12.890235 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 729/1473] 10.222.0.93 () {54 vars in 966 bytes} [Mon Aug 1 15:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:06:12.895332 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056250a6-c868-412f-85ac-ba670cc18c48 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:12.895617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056250a6-c868-412f-85ac-ba670cc18c48 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:12.895893 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056250a6-c868-412f-85ac-ba670cc18c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:12.972617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-056250a6-c868-412f-85ac-ba670cc18c48 None None] MFA Rules not processed for user `4461a389da5848f5afd9854758707829`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:12.973837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-056250a6-c868-412f-85ac-ba670cc18c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:12.995325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:13.053341 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 745/1474] 10.222.0.93 () {58 vars in 1063 bytes} [Mon Aug 1 15:06:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:13.062399 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad5add-d4c7-477a-a06a-b48436f66dd6 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:13.063935 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-0aad5add-d4c7-477a-a06a-b48436f66dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:13.169311 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad5add-d4c7-477a-a06a-b48436f66dd6 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:13.247432 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f7009-2b37-4b1a-84f4-c24602e2561c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4461a389da5848f5afd9854758707829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:13.248078 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f7009-2b37-4b1a-84f4-c24602e2561c service glance] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:13.248351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f7009-2b37-4b1a-84f4-c24602e2561c service glance] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:13.248707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f7009-2b37-4b1a-84f4-c24602e2561c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:13.250341 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c65f7009-2b37-4b1a-84f4-c24602e2561c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:13.308660 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:13.365425 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f7009-2b37-4b1a-84f4-c24602e2561c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:13.367487 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f7009-2b37-4b1a-84f4-c24602e2561c service glance] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:13.385710 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 730/1475] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:06:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:15.496970 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6b854-eecb-423a-9389-938cc91dea55 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:15.497515 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6b854-eecb-423a-9389-938cc91dea55 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:15.497515 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6b854-eecb-423a-9389-938cc91dea55 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:15.498877 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 746/1476] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:06:15.504712 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262128d-7f29-4fd4-90c6-c6391be07de0 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:15.505471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262128d-7f29-4fd4-90c6-c6391be07de0 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:15.505471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262128d-7f29-4fd4-90c6-c6391be07de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:15.574207 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-f262128d-7f29-4fd4-90c6-c6391be07de0 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:15.575440 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-f262128d-7f29-4fd4-90c6-c6391be07de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:15.680882 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 731/1477] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:15.704951 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6b854-eecb-423a-9389-938cc91dea55 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:15.707850 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e3c6b854-eecb-423a-9389-938cc91dea55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:15.846072 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6b854-eecb-423a-9389-938cc91dea55 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:15.901201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:15.952330 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743473d0-4653-4bae-a924-4b68ad6eff9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:15.952988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743473d0-4653-4bae-a924-4b68ad6eff9e admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:15.953271 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743473d0-4653-4bae-a924-4b68ad6eff9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:15.953547 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743473d0-4653-4bae-a924-4b68ad6eff9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:15.954244 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743473d0-4653-4bae-a924-4b68ad6eff9e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:15.956094 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: WARNING py.warnings [None req-743473d0-4653-4bae-a924-4b68ad6eff9e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:06:15.956094 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: warnings.warn(msg) Aug 01 15:06:15.956094 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: Aug 01 15:06:15.957284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743473d0-4653-4bae-a924-4b68ad6eff9e admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:15.978186 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 747/1478] 10.222.0.93 () {56 vars in 1108 bytes} [Mon Aug 1 15:06:15 2022] GET /identity/v3/projects => generated 2553 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:06:18.043993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329100d-8310-4307-8b9b-993a48ba32bc None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:18.044596 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329100d-8310-4307-8b9b-993a48ba32bc None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:18.044596 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329100d-8310-4307-8b9b-993a48ba32bc None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:18.045917 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 732/1479] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:06:18.051940 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54eff28-62a3-43d0-bf0b-853a1a509cb2 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:18.052566 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54eff28-62a3-43d0-bf0b-853a1a509cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:18.052566 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54eff28-62a3-43d0-bf0b-853a1a509cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:18.127054 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-f54eff28-62a3-43d0-bf0b-853a1a509cb2 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:18.128323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f54eff28-62a3-43d0-bf0b-853a1a509cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:18.242060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 748/1480] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:18.287947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329100d-8310-4307-8b9b-993a48ba32bc None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:18.290153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a329100d-8310-4307-8b9b-993a48ba32bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:18.300218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:18.470456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329100d-8310-4307-8b9b-993a48ba32bc None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:18.557962 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:18.600900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117a4e8-ff1b-4a67-9f2b-dfe7eeef0e9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:18.601777 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117a4e8-ff1b-4a67-9f2b-dfe7eeef0e9f service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:18.602156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117a4e8-ff1b-4a67-9f2b-dfe7eeef0e9f service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:18.602445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117a4e8-ff1b-4a67-9f2b-dfe7eeef0e9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:18.604200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b117a4e8-ff1b-4a67-9f2b-dfe7eeef0e9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:18.720685 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117a4e8-ff1b-4a67-9f2b-dfe7eeef0e9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:18.722498 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117a4e8-ff1b-4a67-9f2b-dfe7eeef0e9f service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:18.744796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 733/1481] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:06:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:21.087515 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5061d-bd74-4233-92c8-9819496a9e7d None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:21.088202 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5061d-bd74-4233-92c8-9819496a9e7d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:21.088202 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5061d-bd74-4233-92c8-9819496a9e7d None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:21.089887 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 749/1482] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:06:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:06:21.096423 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5425fc-c224-4d62-8992-719e69fe0a31 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:21.099009 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5425fc-c224-4d62-8992-719e69fe0a31 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:21.099009 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5425fc-c224-4d62-8992-719e69fe0a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:21.176053 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-ab5425fc-c224-4d62-8992-719e69fe0a31 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:21.177353 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ab5425fc-c224-4d62-8992-719e69fe0a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:21.229480 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:21.291929 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 734/1483] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:21.329182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5061d-bd74-4233-92c8-9819496a9e7d None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:21.331087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3af5061d-bd74-4233-92c8-9819496a9e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:21.356293 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:21.442651 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5061d-bd74-4233-92c8-9819496a9e7d None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:21.537485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d81b4-df8f-4c73-bb24-11a972ea7dd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:21.538115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d81b4-df8f-4c73-bb24-11a972ea7dd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:21.538933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d81b4-df8f-4c73-bb24-11a972ea7dd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:21.539207 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d81b4-df8f-4c73-bb24-11a972ea7dd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:21.540969 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-063d81b4-df8f-4c73-bb24-11a972ea7dd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:21.642003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:21.681230 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063d81b4-df8f-4c73-bb24-11a972ea7dd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:21.682712 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063d81b4-df8f-4c73-bb24-11a972ea7dd7 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:21.703719 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 750/1484] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:06:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:23.798977 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70732ef5-a5c0-40c9-b44f-114af5755e21 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:23.799518 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70732ef5-a5c0-40c9-b44f-114af5755e21 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:23.799518 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70732ef5-a5c0-40c9-b44f-114af5755e21 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:23.800892 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 735/1485] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:06:23.806370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de684ecb-e7b1-464d-8e3a-b291e5b98e70 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:23.807018 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de684ecb-e7b1-464d-8e3a-b291e5b98e70 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:23.807018 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de684ecb-e7b1-464d-8e3a-b291e5b98e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:23.996157 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-de684ecb-e7b1-464d-8e3a-b291e5b98e70 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:23.997420 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-de684ecb-e7b1-464d-8e3a-b291e5b98e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:24.098924 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 751/1486] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:06:24.138402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70732ef5-a5c0-40c9-b44f-114af5755e21 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:24.140019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-70732ef5-a5c0-40c9-b44f-114af5755e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:24.252391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70732ef5-a5c0-40c9-b44f-114af5755e21 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:24.328363 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:24.330382 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc788149-7a56-4ee7-8f9e-3d5f774ee760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:24.331143 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc788149-7a56-4ee7-8f9e-3d5f774ee760 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:24.331678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc788149-7a56-4ee7-8f9e-3d5f774ee760 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:24.332224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc788149-7a56-4ee7-8f9e-3d5f774ee760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:24.334689 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-dc788149-7a56-4ee7-8f9e-3d5f774ee760 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:24.451895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc788149-7a56-4ee7-8f9e-3d5f774ee760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:24.453288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc788149-7a56-4ee7-8f9e-3d5f774ee760 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:24.472644 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 736/1487] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:06:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:06:26.691398 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4497259-8a78-4a7e-b1f7-138cf3989450 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:26.691944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4497259-8a78-4a7e-b1f7-138cf3989450 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:26.691944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4497259-8a78-4a7e-b1f7-138cf3989450 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:26.693385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 752/1488] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:06:26.698923 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e61345-2f65-421c-af75-cc13aa04a3f9 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:26.699402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e61345-2f65-421c-af75-cc13aa04a3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:26.699502 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e61345-2f65-421c-af75-cc13aa04a3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:26.767856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-16e61345-2f65-421c-af75-cc13aa04a3f9 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:26.769016 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-16e61345-2f65-421c-af75-cc13aa04a3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:26.823762 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:26.873800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 737/1489] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:26.914766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4497259-8a78-4a7e-b1f7-138cf3989450 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:26.916693 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f4497259-8a78-4a7e-b1f7-138cf3989450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:26.957173 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:27.032704 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4497259-8a78-4a7e-b1f7-138cf3989450 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:27.113999 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08299da-b43b-4beb-9a50-475a9459a842 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:27.113999 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08299da-b43b-4beb-9a50-475a9459a842 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:27.113999 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08299da-b43b-4beb-9a50-475a9459a842 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:27.113999 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08299da-b43b-4beb-9a50-475a9459a842 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:27.116015 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-c08299da-b43b-4beb-9a50-475a9459a842 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:27.203659 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:27.241138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08299da-b43b-4beb-9a50-475a9459a842 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:27.242544 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08299da-b43b-4beb-9a50-475a9459a842 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:27.262797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 753/1490] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:06:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:29.616367 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d268e-515b-4321-ae72-bc14b76486ab None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:29.617511 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d268e-515b-4321-ae72-bc14b76486ab None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:29.617641 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d268e-515b-4321-ae72-bc14b76486ab None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:29.619153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 738/1491] 10.222.0.93 () {54 vars in 926 bytes} [Mon Aug 1 15:06:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:06:29.624670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e046868-b984-43c0-94bb-e568c2422c55 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:29.625174 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e046868-b984-43c0-94bb-e568c2422c55 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:29.625296 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e046868-b984-43c0-94bb-e568c2422c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:29.693422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-1e046868-b984-43c0-94bb-e568c2422c55 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:29.694643 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-1e046868-b984-43c0-94bb-e568c2422c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:29.803360 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 754/1492] 10.222.0.93 () {58 vars in 1023 bytes} [Mon Aug 1 15:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:29.843537 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d268e-515b-4321-ae72-bc14b76486ab None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:29.845111 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-cf8d268e-515b-4321-ae72-bc14b76486ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:29.953443 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d268e-515b-4321-ae72-bc14b76486ab None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:30.034773 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff897cb-6b9f-4059-89f3-a2d242362428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:30.035505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff897cb-6b9f-4059-89f3-a2d242362428 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:30.035758 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff897cb-6b9f-4059-89f3-a2d242362428 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:30.036027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff897cb-6b9f-4059-89f3-a2d242362428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:30.038078 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-fff897cb-6b9f-4059-89f3-a2d242362428 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:30.049806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:30.177161 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff897cb-6b9f-4059-89f3-a2d242362428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:30.178669 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff897cb-6b9f-4059-89f3-a2d242362428 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:30.198444 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 739/1493] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:06:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:39.801536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=82 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:40.196238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=50 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:14.726213 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7188b86-2f84-45f3-80c4-8d4bf2ccbe63 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:14.726926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7188b86-2f84-45f3-80c4-8d4bf2ccbe63 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:14.727084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7188b86-2f84-45f3-80c4-8d4bf2ccbe63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:14.854361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-f7188b86-2f84-45f3-80c4-8d4bf2ccbe63 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:08:14.855782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f7188b86-2f84-45f3-80c4-8d4bf2ccbe63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:14.975029 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 755/1494] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:08:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:08:14.984287 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8eeed7-b116-48c9-a442-080acb6f9042 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:14.984994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8eeed7-b116-48c9-a442-080acb6f9042 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:14.984994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8eeed7-b116-48c9-a442-080acb6f9042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:15.085852 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-3b8eeed7-b116-48c9-a442-080acb6f9042 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:08:15.087126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-3b8eeed7-b116-48c9-a442-080acb6f9042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:15.225129 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 740/1495] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:08:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:08:15.230129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7188b86-2f84-45f3-80c4-8d4bf2ccbe63 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:08:15.231443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f7188b86-2f84-45f3-80c4-8d4bf2ccbe63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:15.246011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7188b86-2f84-45f3-80c4-8d4bf2ccbe63 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:08:15.323240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:15.365030 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c1564-88ca-4f3b-b80b-c86f0315f7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:08:15.365712 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c1564-88ca-4f3b-b80b-c86f0315f7be admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:15.366105 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c1564-88ca-4f3b-b80b-c86f0315f7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:15.366419 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c1564-88ca-4f3b-b80b-c86f0315f7be admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:15.367182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c1564-88ca-4f3b-b80b-c86f0315f7be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:08:15.367517 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c1564-88ca-4f3b-b80b-c86f0315f7be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:08:15.368948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: WARNING py.warnings [None req-de7c1564-88ca-4f3b-b80b-c86f0315f7be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:08:15.368948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: warnings.warn(msg) Aug 01 15:08:15.368948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: Aug 01 15:08:15.369843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c1564-88ca-4f3b-b80b-c86f0315f7be admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:08:15.383851 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 756/1496] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:08:15.393112 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8eeed7-b116-48c9-a442-080acb6f9042 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:08:15.404123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8eeed7-b116-48c9-a442-080acb6f9042 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:08:15.480099 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:15.524399 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbff19-7183-4888-a36d-5f0cd46ac3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:08:15.525246 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbff19-7183-4888-a36d-5f0cd46ac3e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:15.525691 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbff19-7183-4888-a36d-5f0cd46ac3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:15.526063 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbff19-7183-4888-a36d-5f0cd46ac3e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:15.526905 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbff19-7183-4888-a36d-5f0cd46ac3e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:08:15.528817 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: WARNING py.warnings [None req-61bbff19-7183-4888-a36d-5f0cd46ac3e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:08:15.528817 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: warnings.warn(msg) Aug 01 15:08:15.528817 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: Aug 01 15:08:15.530033 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbff19-7183-4888-a36d-5f0cd46ac3e2 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:08:15.597991 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 741/1497] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:08:15 2022] POST /identity/v3/projects => generated 377 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:08:15.604426 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c1564-88ca-4f3b-b80b-c86f0315f7be admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:08:15.607621 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-de7c1564-88ca-4f3b-b80b-c86f0315f7be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:15.657217 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c1564-88ca-4f3b-b80b-c86f0315f7be admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:08:15.859496 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890aba48-76da-4b7e-b3ef-c07cfe9478b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:08:15.860650 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890aba48-76da-4b7e-b3ef-c07cfe9478b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:15.860650 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890aba48-76da-4b7e-b3ef-c07cfe9478b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:15.860961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890aba48-76da-4b7e-b3ef-c07cfe9478b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:15.861439 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890aba48-76da-4b7e-b3ef-c07cfe9478b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:08:15.863051 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: WARNING py.warnings [None req-890aba48-76da-4b7e-b3ef-c07cfe9478b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:08:15.863051 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: warnings.warn(msg) Aug 01 15:08:15.863051 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: Aug 01 15:08:15.865866 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890aba48-76da-4b7e-b3ef-c07cfe9478b9 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:08:15.945577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 757/1498] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:08:15 2022] POST /identity/v3/users => generated 289 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:08:15.951454 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbff19-7183-4888-a36d-5f0cd46ac3e2 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:08:15.953812 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-61bbff19-7183-4888-a36d-5f0cd46ac3e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:16.049855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbff19-7183-4888-a36d-5f0cd46ac3e2 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:08:16.202451 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:16.219946 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0932f-ab4a-4e9b-9a50-55232d71e959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:08:16.220672 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0932f-ab4a-4e9b-9a50-55232d71e959 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:16.221058 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0932f-ab4a-4e9b-9a50-55232d71e959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:16.221480 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0932f-ab4a-4e9b-9a50-55232d71e959 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:16.222323 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0932f-ab4a-4e9b-9a50-55232d71e959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:08:16.223441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: WARNING py.warnings [None req-79a0932f-ab4a-4e9b-9a50-55232d71e959 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:08:16.223441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: warnings.warn(msg) Aug 01 15:08:16.223441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: Aug 01 15:08:16.224649 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0932f-ab4a-4e9b-9a50-55232d71e959 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:08:16.242709 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 742/1499] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:08:16.247711 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890aba48-76da-4b7e-b3ef-c07cfe9478b9 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:08:16.250866 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-890aba48-76da-4b7e-b3ef-c07cfe9478b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:16.272122 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890aba48-76da-4b7e-b3ef-c07cfe9478b9 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:08:16.318226 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:16.399762 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b53f58-ee72-458f-a4b1-1bafece3dd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:08:16.400375 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b53f58-ee72-458f-a4b1-1bafece3dd6d admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:16.400677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b53f58-ee72-458f-a4b1-1bafece3dd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:16.401396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b53f58-ee72-458f-a4b1-1bafece3dd6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:16.401636 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b53f58-ee72-458f-a4b1-1bafece3dd6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:08:16.402661 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: WARNING py.warnings [None req-99b53f58-ee72-458f-a4b1-1bafece3dd6d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:08:16.402661 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: warnings.warn(msg) Aug 01 15:08:16.402661 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: Aug 01 15:08:16.403537 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b53f58-ee72-458f-a4b1-1bafece3dd6d admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:08:16.417313 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 758/1500] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:08:16.421750 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0932f-ab4a-4e9b-9a50-55232d71e959 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:08:16.423379 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-79a0932f-ab4a-4e9b-9a50-55232d71e959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:16.463574 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0932f-ab4a-4e9b-9a50-55232d71e959 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:08:16.588105 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b63cf-2958-4bdb-a93b-b198c6e28d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:08:16.588833 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b63cf-2958-4bdb-a93b-b198c6e28d1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:16.589114 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b63cf-2958-4bdb-a93b-b198c6e28d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:16.589391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b63cf-2958-4bdb-a93b-b198c6e28d1b admin admin] PATH_INFO: `/v3/projects/e3569439b80a4e478001cc792f8e23d0/users/7079550a6f8e4a6a9d1142ad05978f04/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:16.619559 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b63cf-2958-4bdb-a93b-b198c6e28d1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3569439b80a4e478001cc792f8e23d0, user_id=7079550a6f8e4a6a9d1142ad05978f04, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:08:16.621101 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: WARNING py.warnings [None req-bb6b63cf-2958-4bdb-a93b-b198c6e28d1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:08:16.621101 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: warnings.warn(msg) Aug 01 15:08:16.621101 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: Aug 01 15:08:16.623203 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b63cf-2958-4bdb-a93b-b198c6e28d1b admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:08:16.655617 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 743/1501] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:08:16 2022] PUT /identity/v3/projects/e3569439b80a4e478001cc792f8e23d0/users/7079550a6f8e4a6a9d1142ad05978f04/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:08:16.669838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c701f1-540f-4348-bea3-124a12ffac12 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:16.670363 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c701f1-540f-4348-bea3-124a12ffac12 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:16.670947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c701f1-540f-4348-bea3-124a12ffac12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:16.766586 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-43c701f1-540f-4348-bea3-124a12ffac12 None None] MFA Rules not processed for user `7079550a6f8e4a6a9d1142ad05978f04`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:08:16.767894 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-43c701f1-540f-4348-bea3-124a12ffac12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:16.870624 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 759/1502] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:08:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:08:16.884593 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2fadb-d020-4659-b195-8f0f9718703e None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:16.884892 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2fadb-d020-4659-b195-8f0f9718703e None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:16.885176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2fadb-d020-4659-b195-8f0f9718703e None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:16.886630 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 744/1503] 10.222.0.93 () {54 vars in 964 bytes} [Mon Aug 1 15:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:08:16.895304 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbe3d4-bde9-468a-b006-a5cff365a6fc None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:16.895757 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbe3d4-bde9-468a-b006-a5cff365a6fc None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:16.896184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbe3d4-bde9-468a-b006-a5cff365a6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:16.936493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:17.011491 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-3bcbe3d4-bde9-468a-b006-a5cff365a6fc None None] MFA Rules not processed for user `67bf3011e2bd47a3b5cf5e0b1173f065`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:08:17.012785 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3bcbe3d4-bde9-468a-b006-a5cff365a6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:17.118201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 760/1504] 10.222.0.93 () {58 vars in 1061 bytes} [Mon Aug 1 15:08:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:08:17.126863 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2fadb-d020-4659-b195-8f0f9718703e None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:08:17.128510 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-72f2fadb-d020-4659-b195-8f0f9718703e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:17.196523 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:17.298653 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2fadb-d020-4659-b195-8f0f9718703e None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:08:17.456096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db8fbe-ff7f-4329-8a3e-a951e0ec352b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:08:17.457602 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db8fbe-ff7f-4329-8a3e-a951e0ec352b service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:17.458111 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db8fbe-ff7f-4329-8a3e-a951e0ec352b service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:17.458615 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db8fbe-ff7f-4329-8a3e-a951e0ec352b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:17.461937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b8db8fbe-ff7f-4329-8a3e-a951e0ec352b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:17.637813 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db8fbe-ff7f-4329-8a3e-a951e0ec352b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:08:17.640630 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db8fbe-ff7f-4329-8a3e-a951e0ec352b service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:08:17.648401 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:17.678562 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 745/1505] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:08:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:08:17.742187 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b2e1e-b268-4ccc-a933-f0d66136c93d None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:17.742526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b2e1e-b268-4ccc-a933-f0d66136c93d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:17.742919 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b2e1e-b268-4ccc-a933-f0d66136c93d None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:17.744293 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 761/1506] 10.222.0.93 () {52 vars in 830 bytes} [Mon Aug 1 15:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:08:17.749221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db8fbe-ff7f-4329-8a3e-a951e0ec352b service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:08:17.750635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b8db8fbe-ff7f-4329-8a3e-a951e0ec352b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:17.890430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db8fbe-ff7f-4329-8a3e-a951e0ec352b service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:08:18.036796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cca8ce-0c1d-454b-999d-f7149722bcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:08:18.037335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cca8ce-0c1d-454b-999d-f7149722bcdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:18.037751 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cca8ce-0c1d-454b-999d-f7149722bcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:18.038037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cca8ce-0c1d-454b-999d-f7149722bcdd admin admin] PATH_INFO: `/v3/users/7079550a6f8e4a6a9d1142ad05978f04` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:18.055060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cca8ce-0c1d-454b-999d-f7149722bcdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7079550a6f8e4a6a9d1142ad05978f04)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:08:18.056132 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: WARNING py.warnings [None req-b1cca8ce-0c1d-454b-999d-f7149722bcdd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:08:18.056132 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: warnings.warn(msg) Aug 01 15:08:18.056132 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: Aug 01 15:08:18.057586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cca8ce-0c1d-454b-999d-f7149722bcdd admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:08:18.072394 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:18.156378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-b1cca8ce-0c1d-454b-999d-f7149722bcdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7079550a6f8e4a6a9d1142ad05978f04'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:08:18.157471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-b1cca8ce-0c1d-454b-999d-f7149722bcdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7079550a6f8e4a6a9d1142ad05978f04'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:08:18.172054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-b1cca8ce-0c1d-454b-999d-f7149722bcdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7079550a6f8e4a6a9d1142ad05978f04'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:08:18.172993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-b1cca8ce-0c1d-454b-999d-f7149722bcdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7079550a6f8e4a6a9d1142ad05978f04'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:08:18.209798 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-b1cca8ce-0c1d-454b-999d-f7149722bcdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7079550a6f8e4a6a9d1142ad05978f04'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:08:18.242119 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 746/1507] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:08:17 2022] DELETE /identity/v3/users/7079550a6f8e4a6a9d1142ad05978f04 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:08:18.248324 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e027142-50c4-4ea3-82f9-31789dee7132 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:18.248953 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e027142-50c4-4ea3-82f9-31789dee7132 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:18.249085 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e027142-50c4-4ea3-82f9-31789dee7132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:18.337600 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-7e027142-50c4-4ea3-82f9-31789dee7132 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:08:18.339792 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-7e027142-50c4-4ea3-82f9-31789dee7132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:18.446232 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:18.469575 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 762/1508] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:08:18.486852 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cca8ce-0c1d-454b-999d-f7149722bcdd admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:08:18.489734 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b1cca8ce-0c1d-454b-999d-f7149722bcdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:18.657282 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cca8ce-0c1d-454b-999d-f7149722bcdd admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:08:18.736694 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:18.763628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4566fd-d54d-442a-81f8-21b72e77ba49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:08:18.764324 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4566fd-d54d-442a-81f8-21b72e77ba49 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:18.764661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4566fd-d54d-442a-81f8-21b72e77ba49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:18.764941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4566fd-d54d-442a-81f8-21b72e77ba49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:18.766760 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-0f4566fd-d54d-442a-81f8-21b72e77ba49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:18.940644 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4566fd-d54d-442a-81f8-21b72e77ba49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:08:18.942059 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4566fd-d54d-442a-81f8-21b72e77ba49 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:08:18.972302 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 747/1509] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:08:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:08:19.518372 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e027142-50c4-4ea3-82f9-31789dee7132 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:08:19.520569 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-7e027142-50c4-4ea3-82f9-31789dee7132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:19.538718 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e027142-50c4-4ea3-82f9-31789dee7132 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:08:19.718562 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282f6e6-3d36-4428-bb97-b0f0f0a78938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:08:19.719329 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282f6e6-3d36-4428-bb97-b0f0f0a78938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:19.719679 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282f6e6-3d36-4428-bb97-b0f0f0a78938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:19.720016 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282f6e6-3d36-4428-bb97-b0f0f0a78938 admin admin] PATH_INFO: `/v3/projects/e3569439b80a4e478001cc792f8e23d0` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:19.738774 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282f6e6-3d36-4428-bb97-b0f0f0a78938 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3569439b80a4e478001cc792f8e23d0)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:08:19.740101 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: WARNING py.warnings [None req-f282f6e6-3d36-4428-bb97-b0f0f0a78938 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:08:19.740101 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: warnings.warn(msg) Aug 01 15:08:19.740101 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: Aug 01 15:08:19.742609 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282f6e6-3d36-4428-bb97-b0f0f0a78938 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:08:19.854217 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f282f6e6-3d36-4428-bb97-b0f0f0a78938 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3569439b80a4e478001cc792f8e23d0', 'request_id': 'req-f282f6e6-3d36-4428-bb97-b0f0f0a78938'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:08:19.873056 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f282f6e6-3d36-4428-bb97-b0f0f0a78938 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3569439b80a4e478001cc792f8e23d0', 'request_id': 'req-f282f6e6-3d36-4428-bb97-b0f0f0a78938'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:08:19.885797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f282f6e6-3d36-4428-bb97-b0f0f0a78938 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3569439b80a4e478001cc792f8e23d0', 'request_id': 'req-f282f6e6-3d36-4428-bb97-b0f0f0a78938'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:08:19.942530 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f282f6e6-3d36-4428-bb97-b0f0f0a78938 admin admin] The token cache is being invalidate because project e3569439b80a4e478001cc792f8e23d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:08:19.943131 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f282f6e6-3d36-4428-bb97-b0f0f0a78938 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:08:19.943427 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f282f6e6-3d36-4428-bb97-b0f0f0a78938 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:08:19.945108 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 763/1510] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:08:19 2022] DELETE /identity/v3/projects/e3569439b80a4e478001cc792f8e23d0 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:08:28.966621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=62 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:29.939596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:12.799573 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de8266-390e-4ee6-9d81-b2098d86eac9 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:12.800598 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de8266-390e-4ee6-9d81-b2098d86eac9 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:12.800598 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de8266-390e-4ee6-9d81-b2098d86eac9 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:12.802319 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:10:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:12.811654 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de8f28-b353-432f-9381-df73d8713e61 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:12.812591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de8f28-b353-432f-9381-df73d8713e61 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:12.812762 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de8f28-b353-432f-9381-df73d8713e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:12.960379 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-b8de8f28-b353-432f-9381-df73d8713e61 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:12.961747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b8de8f28-b353-432f-9381-df73d8713e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:13.172672 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:10:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:10:13.344040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de8266-390e-4ee6-9d81-b2098d86eac9 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:13.348077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-52de8266-390e-4ee6-9d81-b2098d86eac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:13.562744 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de8266-390e-4ee6-9d81-b2098d86eac9 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:13.571306 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9493bf9-6146-41b5-8971-386a21f48a93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:13.572319 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9493bf9-6146-41b5-8971-386a21f48a93 service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:13.572764 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9493bf9-6146-41b5-8971-386a21f48a93 service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:13.573052 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9493bf9-6146-41b5-8971-386a21f48a93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:13.587866 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9493bf9-6146-41b5-8971-386a21f48a93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:13.591235 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9493bf9-6146-41b5-8971-386a21f48a93 service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:13.611087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 749/1513] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:10:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:10:14.482072 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501c451-b35d-41e1-a753-1aaa1c7e4a4b None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:14.482739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501c451-b35d-41e1-a753-1aaa1c7e4a4b None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:14.482739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501c451-b35d-41e1-a753-1aaa1c7e4a4b None None] PATH_INFO: `/v3` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:14.484318 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 765/1514] 10.222.0.61 () {54 vars in 925 bytes} [Mon Aug 1 15:10:14 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:14.511510 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d516703-2b5d-49a5-ba8e-f3eb3569f86d None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:14.512330 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d516703-2b5d-49a5-ba8e-f3eb3569f86d None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:14.512629 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d516703-2b5d-49a5-ba8e-f3eb3569f86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:14.632980 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdb69a-a797-4e61-8e0e-bd6e55e46d2e None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:14.633513 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdb69a-a797-4e61-8e0e-bd6e55e46d2e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:14.634024 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdb69a-a797-4e61-8e0e-bd6e55e46d2e None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:14.636064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 766/1515] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:10:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:10:14.640128 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-9d516703-2b5d-49a5-ba8e-f3eb3569f86d None None] MFA Rules not processed for user `a07e3e25e6a8466fb237dcb836aecda2`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:14.642810 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-9d516703-2b5d-49a5-ba8e-f3eb3569f86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:14.643773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2cbfd-ceb9-4bbe-9abe-b69899f53ef8 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:14.644220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2cbfd-ceb9-4bbe-9abe-b69899f53ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:14.644694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2cbfd-ceb9-4bbe-9abe-b69899f53ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:14.675192 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-efd2cbfd-ceb9-4bbe-9abe-b69899f53ef8 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:14.676725 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-efd2cbfd-ceb9-4bbe-9abe-b69899f53ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:14.686694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 767/1516] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:10:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:10:14.737384 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:14.745544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 750/1517] 10.222.0.61 () {58 vars in 1015 bytes} [Mon Aug 1 15:10:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:10:14.874580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2cbfd-ceb9-4bbe-9abe-b69899f53ef8 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:14.888804 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2cbfd-ceb9-4bbe-9abe-b69899f53ef8 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:14.896297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca8a4f-3f0f-4a04-96c7-1fc1bf2db6fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:14.897221 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca8a4f-3f0f-4a04-96c7-1fc1bf2db6fc service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:14.897625 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca8a4f-3f0f-4a04-96c7-1fc1bf2db6fc service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:14.898008 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca8a4f-3f0f-4a04-96c7-1fc1bf2db6fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:14.910331 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca8a4f-3f0f-4a04-96c7-1fc1bf2db6fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:14.912214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca8a4f-3f0f-4a04-96c7-1fc1bf2db6fc service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:14.925511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 768/1518] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:10:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:10:18.796572 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3615e-63db-4077-ba13-8fff598af5d9 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:18.797719 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3615e-63db-4077-ba13-8fff598af5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:18.798211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3615e-63db-4077-ba13-8fff598af5d9 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:18.800885 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 751/1519] 10.222.0.93 () {54 vars in 961 bytes} [Mon Aug 1 15:10:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:18.810113 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f808fdb-af54-48d8-bf5b-943d38e513a5 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:18.810772 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f808fdb-af54-48d8-bf5b-943d38e513a5 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:18.810772 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f808fdb-af54-48d8-bf5b-943d38e513a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:18.834066 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-5f808fdb-af54-48d8-bf5b-943d38e513a5 None None] MFA Rules not processed for user `a07e3e25e6a8466fb237dcb836aecda2`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:18.835435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5f808fdb-af54-48d8-bf5b-943d38e513a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:18.845146 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 769/1520] 10.222.0.93 () {58 vars in 1058 bytes} [Mon Aug 1 15:10:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:10:18.853724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3615e-63db-4077-ba13-8fff598af5d9 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:18.869856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3615e-63db-4077-ba13-8fff598af5d9 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:18.881283 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3defb-2c01-4f91-ab1e-1709e49720ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07e3e25e6a8466fb237dcb836aecda2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:18.882041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3defb-2c01-4f91-ab1e-1709e49720ca service placement] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:18.882421 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3defb-2c01-4f91-ab1e-1709e49720ca service placement] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:18.882800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3defb-2c01-4f91-ab1e-1709e49720ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:18.893852 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3defb-2c01-4f91-ab1e-1709e49720ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:18.895778 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3defb-2c01-4f91-ab1e-1709e49720ca service placement] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:18.907982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 752/1521] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:10:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:10:26.341666 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510cb159-d786-47f0-a98a-63ce6f86a161 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:26.342575 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510cb159-d786-47f0-a98a-63ce6f86a161 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:26.342575 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510cb159-d786-47f0-a98a-63ce6f86a161 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:26.344842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 770/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:10:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:26.351659 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f4ef0-7a72-4b18-b909-8915b5193afc None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:26.352554 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f4ef0-7a72-4b18-b909-8915b5193afc None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:26.352988 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f4ef0-7a72-4b18-b909-8915b5193afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:26.376851 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-5c7f4ef0-7a72-4b18-b909-8915b5193afc None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:26.378319 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-5c7f4ef0-7a72-4b18-b909-8915b5193afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:26.389238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 753/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:10:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:10:26.582876 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510cb159-d786-47f0-a98a-63ce6f86a161 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:26.600102 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510cb159-d786-47f0-a98a-63ce6f86a161 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:26.606411 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a000ca74-9325-4cb1-8072-af0d64ec9f31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:26.607114 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000ca74-9325-4cb1-8072-af0d64ec9f31 service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:26.607428 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000ca74-9325-4cb1-8072-af0d64ec9f31 service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:26.607731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000ca74-9325-4cb1-8072-af0d64ec9f31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:26.618969 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a000ca74-9325-4cb1-8072-af0d64ec9f31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:26.620857 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a000ca74-9325-4cb1-8072-af0d64ec9f31 service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:26.632635 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 771/1524] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:10:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:10:27.720597 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3bed8-bcd8-4da0-ac0e-3239b18eb40b None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:27.721445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3bed8-bcd8-4da0-ac0e-3239b18eb40b None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:27.721445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3bed8-bcd8-4da0-ac0e-3239b18eb40b None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:27.722997 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 754/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:10:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:27.730188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9706b-9ed8-4f9d-9cfc-1754c937fb5b None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:27.731356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9706b-9ed8-4f9d-9cfc-1754c937fb5b None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:27.731992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9706b-9ed8-4f9d-9cfc-1754c937fb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:27.766343 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-21d9706b-9ed8-4f9d-9cfc-1754c937fb5b None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:27.768272 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-21d9706b-9ed8-4f9d-9cfc-1754c937fb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:27.783014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 772/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:10:28.085640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3bed8-bcd8-4da0-ac0e-3239b18eb40b None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:28.122547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3bed8-bcd8-4da0-ac0e-3239b18eb40b None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:28.129654 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175dc1f6-496f-4fc1-ae7d-fb19964979ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:28.130309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175dc1f6-496f-4fc1-ae7d-fb19964979ae service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:28.130682 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175dc1f6-496f-4fc1-ae7d-fb19964979ae service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:28.130964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175dc1f6-496f-4fc1-ae7d-fb19964979ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:28.143273 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175dc1f6-496f-4fc1-ae7d-fb19964979ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:28.145116 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175dc1f6-496f-4fc1-ae7d-fb19964979ae service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:28.159468 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 755/1527] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:10:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:10:32.457466 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb637447-1b8e-4bee-9e28-33f35c6a906a None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:32.458227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb637447-1b8e-4bee-9e28-33f35c6a906a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:32.458227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb637447-1b8e-4bee-9e28-33f35c6a906a None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:32.459641 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 773/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:32.465874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad7bd1-f0a1-4557-b455-b92d7106727c None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:32.466475 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad7bd1-f0a1-4557-b455-b92d7106727c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:32.466475 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad7bd1-f0a1-4557-b455-b92d7106727c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:32.490866 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-01ad7bd1-f0a1-4557-b455-b92d7106727c None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:32.492485 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-01ad7bd1-f0a1-4557-b455-b92d7106727c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:32.503982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 756/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:10:32.589888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb637447-1b8e-4bee-9e28-33f35c6a906a None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:32.592902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-cb637447-1b8e-4bee-9e28-33f35c6a906a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:32.642921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:32.718220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb637447-1b8e-4bee-9e28-33f35c6a906a None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:32.723583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a9e44-2503-4bd5-98d5-522a6fd71fb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:32.724714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a9e44-2503-4bd5-98d5-522a6fd71fb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:32.725167 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a9e44-2503-4bd5-98d5-522a6fd71fb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:32.725661 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a9e44-2503-4bd5-98d5-522a6fd71fb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:32.743336 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a9e44-2503-4bd5-98d5-522a6fd71fb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:32.746052 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a9e44-2503-4bd5-98d5-522a6fd71fb4 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:32.764591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 774/1530] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:10:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:10:34.916792 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66790dc6-00ab-4acf-82a9-93d0256e857f None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:34.917580 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66790dc6-00ab-4acf-82a9-93d0256e857f None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:34.917580 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66790dc6-00ab-4acf-82a9-93d0256e857f None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:34.918722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 757/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:10:34.929048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a653824-cb43-4e3a-8428-d187a1479574 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:34.930177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a653824-cb43-4e3a-8428-d187a1479574 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:34.930177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a653824-cb43-4e3a-8428-d187a1479574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:34.952010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-8a653824-cb43-4e3a-8428-d187a1479574 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:34.953855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-8a653824-cb43-4e3a-8428-d187a1479574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:34.962947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 775/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:10:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:10:35.008748 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66790dc6-00ab-4acf-82a9-93d0256e857f None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:35.030472 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66790dc6-00ab-4acf-82a9-93d0256e857f None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:35.035460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce01d6f-b268-4c3f-8ace-021a3a489ec4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:35.036316 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce01d6f-b268-4c3f-8ace-021a3a489ec4 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:35.036689 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce01d6f-b268-4c3f-8ace-021a3a489ec4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:35.037003 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce01d6f-b268-4c3f-8ace-021a3a489ec4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:35.049003 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce01d6f-b268-4c3f-8ace-021a3a489ec4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:35.050673 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce01d6f-b268-4c3f-8ace-021a3a489ec4 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:35.063863 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 758/1533] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:10:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:10:39.017246 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998794f-0834-471f-ab36-4c49d6f06449 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:39.018092 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998794f-0834-471f-ab36-4c49d6f06449 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:39.018263 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998794f-0834-471f-ab36-4c49d6f06449 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:39.021261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 776/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:10:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:39.027554 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a28403-703c-4b78-a4ca-43fa6948820b None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:39.028228 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a28403-703c-4b78-a4ca-43fa6948820b None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:39.028228 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a28403-703c-4b78-a4ca-43fa6948820b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:39.057921 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-51a28403-703c-4b78-a4ca-43fa6948820b None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:39.060169 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-51a28403-703c-4b78-a4ca-43fa6948820b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:39.074761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 759/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:10:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:10:39.409054 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998794f-0834-471f-ab36-4c49d6f06449 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:39.411154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5998794f-0834-471f-ab36-4c49d6f06449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:39.508604 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998794f-0834-471f-ab36-4c49d6f06449 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:39.512885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6910e-8bf4-448d-982a-32a1f101272f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:39.513559 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6910e-8bf4-448d-982a-32a1f101272f service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:39.513838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6910e-8bf4-448d-982a-32a1f101272f service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:39.514121 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6910e-8bf4-448d-982a-32a1f101272f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:39.525489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6910e-8bf4-448d-982a-32a1f101272f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:39.527030 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6910e-8bf4-448d-982a-32a1f101272f service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:39.538622 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 777/1536] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:10:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:10:42.094101 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799e4c0-51d3-43d0-a07d-ae24c0213ede None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:42.094910 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799e4c0-51d3-43d0-a07d-ae24c0213ede None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:42.094910 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799e4c0-51d3-43d0-a07d-ae24c0213ede None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:42.097204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 760/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:10:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:10:42.106932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dca74c-4fde-4693-8b47-9b6f5229e09e None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:42.107841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dca74c-4fde-4693-8b47-9b6f5229e09e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:42.108022 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dca74c-4fde-4693-8b47-9b6f5229e09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:42.142689 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-83dca74c-4fde-4693-8b47-9b6f5229e09e None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:42.144968 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-83dca74c-4fde-4693-8b47-9b6f5229e09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:42.161094 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 778/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:10:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:10:42.665000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799e4c0-51d3-43d0-a07d-ae24c0213ede None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:42.667793 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-6799e4c0-51d3-43d0-a07d-ae24c0213ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:42.693049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799e4c0-51d3-43d0-a07d-ae24c0213ede None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:42.699093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f71a44-7919-40e6-8582-98060c98307d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:42.700127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f71a44-7919-40e6-8582-98060c98307d service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:42.700664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f71a44-7919-40e6-8582-98060c98307d service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:42.701150 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f71a44-7919-40e6-8582-98060c98307d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:42.719683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f71a44-7919-40e6-8582-98060c98307d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:42.723099 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f71a44-7919-40e6-8582-98060c98307d service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:42.742984 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 761/1539] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:10:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:10:52.133643 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=66 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:52.733867 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=34 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:29.769393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585c843-74ee-4fe1-8d15-4574fdd66993 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:29.770679 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2d036-1cb3-4fb3-bd43-ccfb0c13cf7e None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:29.771387 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585c843-74ee-4fe1-8d15-4574fdd66993 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:29.771569 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2d036-1cb3-4fb3-bd43-ccfb0c13cf7e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:29.771730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585c843-74ee-4fe1-8d15-4574fdd66993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:29.771888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2d036-1cb3-4fb3-bd43-ccfb0c13cf7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:29.828181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-1585c843-74ee-4fe1-8d15-4574fdd66993 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:29.830435 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-1585c843-74ee-4fe1-8d15-4574fdd66993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:29.832491 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-e3c2d036-1cb3-4fb3-bd43-ccfb0c13cf7e None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:29.834723 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e3c2d036-1cb3-4fb3-bd43-ccfb0c13cf7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:29.865253 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 779/1540] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:29.865937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 762/1541] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:12:29.879321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfdcc9-90a9-47da-a575-f47693b9a7ad None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:29.879848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfdcc9-90a9-47da-a575-f47693b9a7ad None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:29.880249 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfdcc9-90a9-47da-a575-f47693b9a7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:29.889864 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c4dbf-7b9a-43a9-b8db-ccdc8f268098 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:29.890437 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c4dbf-7b9a-43a9-b8db-ccdc8f268098 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:29.890813 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c4dbf-7b9a-43a9-b8db-ccdc8f268098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:29.933871 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-89bfdcc9-90a9-47da-a575-f47693b9a7ad None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:29.936102 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-89bfdcc9-90a9-47da-a575-f47693b9a7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:29.959159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-ac5c4dbf-7b9a-43a9-b8db-ccdc8f268098 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:29.961292 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ac5c4dbf-7b9a-43a9-b8db-ccdc8f268098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:29.970816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 780/1542] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:29.979370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda1fdf-e9d1-4c34-8af7-6ba5763f75c2 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:29.979892 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda1fdf-e9d1-4c34-8af7-6ba5763f75c2 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:29.980298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda1fdf-e9d1-4c34-8af7-6ba5763f75c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.003779 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 763/1543] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:12:30.006868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d77dd-2bbc-485b-a7ee-2828b5be7b99 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.007367 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d77dd-2bbc-485b-a7ee-2828b5be7b99 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.007853 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d77dd-2bbc-485b-a7ee-2828b5be7b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.054540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-5fda1fdf-e9d1-4c34-8af7-6ba5763f75c2 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:30.057922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5fda1fdf-e9d1-4c34-8af7-6ba5763f75c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:30.066988 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-701d77dd-2bbc-485b-a7ee-2828b5be7b99 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:30.069280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-701d77dd-2bbc-485b-a7ee-2828b5be7b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:30.080002 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 781/1544] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:30.083987 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ff736-5c66-4158-880b-3b9747bf4dc3 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.084750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ff736-5c66-4158-880b-3b9747bf4dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.085228 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ff736-5c66-4158-880b-3b9747bf4dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.095176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 764/1545] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:30.097237 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d77dd-2bbc-485b-a7ee-2828b5be7b99 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:30.123499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d77dd-2bbc-485b-a7ee-2828b5be7b99 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:30.133177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75bfe7-497f-42c0-ab72-9d3f01c85c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:30.134162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75bfe7-497f-42c0-ab72-9d3f01c85c38 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.134617 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75bfe7-497f-42c0-ab72-9d3f01c85c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.135061 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75bfe7-497f-42c0-ab72-9d3f01c85c38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.135613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-752ff736-5c66-4158-880b-3b9747bf4dc3 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:30.136512 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c75bfe7-497f-42c0-ab72-9d3f01c85c38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:30.137678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c75bfe7-497f-42c0-ab72-9d3f01c85c38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:30.138468 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-752ff736-5c66-4158-880b-3b9747bf4dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:30.139471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: WARNING py.warnings [None req-1c75bfe7-497f-42c0-ab72-9d3f01c85c38 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:12:30.139471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: warnings.warn(msg) Aug 01 15:12:30.139471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: Aug 01 15:12:30.141149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c75bfe7-497f-42c0-ab72-9d3f01c85c38 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:30.155201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 782/1546] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:30.158427 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ff736-5c66-4158-880b-3b9747bf4dc3 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:30.168113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 765/1547] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:30.170941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b486dd7-6a31-4734-862e-545b08157dbb None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.172231 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b486dd7-6a31-4734-862e-545b08157dbb None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.173075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b486dd7-6a31-4734-862e-545b08157dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.191835 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ff736-5c66-4158-880b-3b9747bf4dc3 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:30.207272 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6cd185-f3d9-47dd-8b9f-e20fc2c10d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:30.207272 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cd185-f3d9-47dd-8b9f-e20fc2c10d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.207812 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cd185-f3d9-47dd-8b9f-e20fc2c10d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.208045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cd185-f3d9-47dd-8b9f-e20fc2c10d72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.209669 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6cd185-f3d9-47dd-8b9f-e20fc2c10d72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:30.210220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6cd185-f3d9-47dd-8b9f-e20fc2c10d72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:30.216588 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6cd185-f3d9-47dd-8b9f-e20fc2c10d72 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:30.231623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-9b486dd7-6a31-4734-862e-545b08157dbb None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:30.235968 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-9b486dd7-6a31-4734-862e-545b08157dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:30.249682 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 783/1548] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:30.252229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae9328-dfef-4ae4-9fed-1a7ccd6020df None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.252782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae9328-dfef-4ae4-9fed-1a7ccd6020df None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.253272 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae9328-dfef-4ae4-9fed-1a7ccd6020df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.259514 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 766/1549] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:30.268514 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d9ea7-d630-41cb-bbcd-1bd8687e422c None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.270169 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d9ea7-d630-41cb-bbcd-1bd8687e422c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.271324 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d9ea7-d630-41cb-bbcd-1bd8687e422c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.304499 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-c3ae9328-dfef-4ae4-9fed-1a7ccd6020df None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:30.304499 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-c3ae9328-dfef-4ae4-9fed-1a7ccd6020df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:30.323334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 784/1550] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:30.325774 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d224c8-c3eb-45c8-b3fa-c960c9fb3f91 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.326269 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d224c8-c3eb-45c8-b3fa-c960c9fb3f91 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.326762 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d224c8-c3eb-45c8-b3fa-c960c9fb3f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.328662 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-1b9d9ea7-d630-41cb-bbcd-1bd8687e422c None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:30.331358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-1b9d9ea7-d630-41cb-bbcd-1bd8687e422c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:30.349265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 767/1551] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:30.351038 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d9ea7-d630-41cb-bbcd-1bd8687e422c None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:30.368252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-29d224c8-c3eb-45c8-b3fa-c960c9fb3f91 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:30.370539 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-29d224c8-c3eb-45c8-b3fa-c960c9fb3f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:30.371599 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d9ea7-d630-41cb-bbcd-1bd8687e422c None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:30.380440 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323a705-f38f-4608-969b-58cd763c30d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:30.381697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323a705-f38f-4608-969b-58cd763c30d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.382261 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323a705-f38f-4608-969b-58cd763c30d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.382740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323a705-f38f-4608-969b-58cd763c30d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.383813 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323a705-f38f-4608-969b-58cd763c30d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:30.386344 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 785/1552] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:30.387922 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323a705-f38f-4608-969b-58cd763c30d7 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:30.393602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d224c8-c3eb-45c8-b3fa-c960c9fb3f91 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:30.416398 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d224c8-c3eb-45c8-b3fa-c960c9fb3f91 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:30.431861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd925a-5fe6-41cc-a30f-a5c73fd09273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:30.431861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd925a-5fe6-41cc-a30f-a5c73fd09273 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.431861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd925a-5fe6-41cc-a30f-a5c73fd09273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.431861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd925a-5fe6-41cc-a30f-a5c73fd09273 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.432700 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd925a-5fe6-41cc-a30f-a5c73fd09273 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:30.432700 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: WARNING py.warnings [None req-1cfd925a-5fe6-41cc-a30f-a5c73fd09273 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:12:30.432700 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: warnings.warn(msg) Aug 01 15:12:30.432700 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: Aug 01 15:12:30.434293 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd925a-5fe6-41cc-a30f-a5c73fd09273 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:30.468439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 768/1553] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:12:30 2022] POST /identity/v3/projects => generated 369 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:30.470863 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d6842-e12c-48ec-93bb-6e411683aa0c None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.471376 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d6842-e12c-48ec-93bb-6e411683aa0c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.471961 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d6842-e12c-48ec-93bb-6e411683aa0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.493214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 786/1554] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:12:30 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:30.493214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d257376-5c25-44c6-837a-be94decb25a3 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.493214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d257376-5c25-44c6-837a-be94decb25a3 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.493214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d257376-5c25-44c6-837a-be94decb25a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.517112 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-8c8d6842-e12c-48ec-93bb-6e411683aa0c None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:30.520504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-8c8d6842-e12c-48ec-93bb-6e411683aa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:30.526223 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-4d257376-5c25-44c6-837a-be94decb25a3 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:30.528323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4d257376-5c25-44c6-837a-be94decb25a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:30.618612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 769/1555] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:30.619743 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d6842-e12c-48ec-93bb-6e411683aa0c None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:30.633367 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d6842-e12c-48ec-93bb-6e411683aa0c None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:30.640313 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ba687-40cc-4a28-9361-4bd7b9d58c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:30.641228 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ba687-40cc-4a28-9361-4bd7b9d58c97 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.641661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ba687-40cc-4a28-9361-4bd7b9d58c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.642050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ba687-40cc-4a28-9361-4bd7b9d58c97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.643263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ba687-40cc-4a28-9361-4bd7b9d58c97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:30.643700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ba687-40cc-4a28-9361-4bd7b9d58c97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:30.645842 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ba687-40cc-4a28-9361-4bd7b9d58c97 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:30.657773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 787/1556] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:30.659170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d257376-5c25-44c6-837a-be94decb25a3 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:30.664534 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 770/1557] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:30.666118 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ba687-40cc-4a28-9361-4bd7b9d58c97 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:30.681300 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d257376-5c25-44c6-837a-be94decb25a3 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:30.687145 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ba687-40cc-4a28-9361-4bd7b9d58c97 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:30.687647 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09a19f-59b1-47ce-b8f1-d82194c19f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:30.687989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09a19f-59b1-47ce-b8f1-d82194c19f06 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.688246 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09a19f-59b1-47ce-b8f1-d82194c19f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.688596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09a19f-59b1-47ce-b8f1-d82194c19f06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.689280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09a19f-59b1-47ce-b8f1-d82194c19f06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:30.689770 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09a19f-59b1-47ce-b8f1-d82194c19f06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:30.691283 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09a19f-59b1-47ce-b8f1-d82194c19f06 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:30.696429 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83779b5-c919-4a94-93e9-dade1ee7dc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:30.697544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83779b5-c919-4a94-93e9-dade1ee7dc9f admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.698022 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83779b5-c919-4a94-93e9-dade1ee7dc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.698470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83779b5-c919-4a94-93e9-dade1ee7dc9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.700249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83779b5-c919-4a94-93e9-dade1ee7dc9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:30.700808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83779b5-c919-4a94-93e9-dade1ee7dc9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:30.704093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83779b5-c919-4a94-93e9-dade1ee7dc9f admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:30.708113 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 788/1558] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:30.709844 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266ff57-0d54-47f9-93fb-2e9e9754deef None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.710227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266ff57-0d54-47f9-93fb-2e9e9754deef None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.710622 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266ff57-0d54-47f9-93fb-2e9e9754deef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.737633 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-5266ff57-0d54-47f9-93fb-2e9e9754deef None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:30.739146 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5266ff57-0d54-47f9-93fb-2e9e9754deef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:30.752964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 771/1559] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:30.752964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83779b5-c919-4a94-93e9-dade1ee7dc9f admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:30.760636 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 789/1560] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:30.761492 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266ff57-0d54-47f9-93fb-2e9e9754deef None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:30.764780 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83779b5-c919-4a94-93e9-dade1ee7dc9f admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:30.773253 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218e1d3-e41a-43dd-97ce-3f4dd6d24211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:30.774366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266ff57-0d54-47f9-93fb-2e9e9754deef None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:30.775060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218e1d3-e41a-43dd-97ce-3f4dd6d24211 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.775272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218e1d3-e41a-43dd-97ce-3f4dd6d24211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.775783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218e1d3-e41a-43dd-97ce-3f4dd6d24211 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.777122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218e1d3-e41a-43dd-97ce-3f4dd6d24211 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:30.779130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: WARNING py.warnings [None req-1218e1d3-e41a-43dd-97ce-3f4dd6d24211 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:12:30.779130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: warnings.warn(msg) Aug 01 15:12:30.779130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: Aug 01 15:12:30.780861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2165690-e2e1-47bc-ad29-47ceb51e74d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:30.781623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218e1d3-e41a-43dd-97ce-3f4dd6d24211 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:30.781828 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2165690-e2e1-47bc-ad29-47ceb51e74d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.782013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2165690-e2e1-47bc-ad29-47ceb51e74d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.782283 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2165690-e2e1-47bc-ad29-47ceb51e74d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.783496 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2165690-e2e1-47bc-ad29-47ceb51e74d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:30.787326 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2165690-e2e1-47bc-ad29-47ceb51e74d0 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:30.824711 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 772/1561] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:12:30 2022] POST /identity/v3/users => generated 285 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:30.824711 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218e1d3-e41a-43dd-97ce-3f4dd6d24211 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:30.829807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 790/1562] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:12:30 2022] POST /identity/v3/users => generated 288 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:30.831376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcce640-fca0-4eca-9a61-97173faf8c87 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.831847 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcce640-fca0-4eca-9a61-97173faf8c87 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.832307 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcce640-fca0-4eca-9a61-97173faf8c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.835595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218e1d3-e41a-43dd-97ce-3f4dd6d24211 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:30.842375 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9cbdc9-8a91-4fe9-8c0b-a915b8f02b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:30.843126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cbdc9-8a91-4fe9-8c0b-a915b8f02b14 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.843569 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cbdc9-8a91-4fe9-8c0b-a915b8f02b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.843934 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cbdc9-8a91-4fe9-8c0b-a915b8f02b14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.844954 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9cbdc9-8a91-4fe9-8c0b-a915b8f02b14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:30.845455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9cbdc9-8a91-4fe9-8c0b-a915b8f02b14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:30.847487 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9cbdc9-8a91-4fe9-8c0b-a915b8f02b14 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:30.864674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 773/1563] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:30.865151 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-0fcce640-fca0-4eca-9a61-97173faf8c87 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:30.865892 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9cbdc9-8a91-4fe9-8c0b-a915b8f02b14 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:30.867241 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0fcce640-fca0-4eca-9a61-97173faf8c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:30.882009 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9cbdc9-8a91-4fe9-8c0b-a915b8f02b14 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:30.882778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 791/1564] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:30.884284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcce640-fca0-4eca-9a61-97173faf8c87 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:30.892693 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d86b9-1ebe-4064-9c98-11ccbb037676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:30.893820 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d86b9-1ebe-4064-9c98-11ccbb037676 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.894269 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d86b9-1ebe-4064-9c98-11ccbb037676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.894759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d86b9-1ebe-4064-9c98-11ccbb037676 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.896025 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d86b9-1ebe-4064-9c98-11ccbb037676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:30.899782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcce640-fca0-4eca-9a61-97173faf8c87 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:30.900273 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d86b9-1ebe-4064-9c98-11ccbb037676 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:30.909571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa36d3a-29a6-4573-bf75-538c88bbc86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:30.910710 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa36d3a-29a6-4573-bf75-538c88bbc86e admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.911212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa36d3a-29a6-4573-bf75-538c88bbc86e admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.911720 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa36d3a-29a6-4573-bf75-538c88bbc86e admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.913070 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa36d3a-29a6-4573-bf75-538c88bbc86e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:30.917217 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa36d3a-29a6-4573-bf75-538c88bbc86e admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:30.957735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 774/1565] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:12:30 2022] POST /identity/v3/projects => generated 403 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:30.957735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d86b9-1ebe-4064-9c98-11ccbb037676 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:30.964244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:30.981441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 792/1566] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:12:30 2022] POST /identity/v3/projects => generated 389 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:30.981441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a471c0-6fc8-4a4d-a239-df10f23d4306 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.981441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a471c0-6fc8-4a4d-a239-df10f23d4306 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.981441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a471c0-6fc8-4a4d-a239-df10f23d4306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.015837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-18a471c0-6fc8-4a4d-a239-df10f23d4306 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:31.017304 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-18a471c0-6fc8-4a4d-a239-df10f23d4306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:31.037386 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:31.084186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d86b9-1ebe-4064-9c98-11ccbb037676 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:31.119511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 793/1567] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:31.120883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a471c0-6fc8-4a4d-a239-df10f23d4306 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:31.134314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a471c0-6fc8-4a4d-a239-df10f23d4306 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:31.142296 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee7561-8b14-4515-ad15-8ddfac4a7adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:31.143326 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee7561-8b14-4515-ad15-8ddfac4a7adc admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:31.143925 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee7561-8b14-4515-ad15-8ddfac4a7adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:31.145556 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee7561-8b14-4515-ad15-8ddfac4a7adc admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.145556 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee7561-8b14-4515-ad15-8ddfac4a7adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:31.147968 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee7561-8b14-4515-ad15-8ddfac4a7adc admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:31.164260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 794/1568] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:31.165820 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee7561-8b14-4515-ad15-8ddfac4a7adc admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:31.188561 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee7561-8b14-4515-ad15-8ddfac4a7adc admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:31.190802 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f11316-7670-46da-83b0-d9644a0952d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:31.191468 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f11316-7670-46da-83b0-d9644a0952d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:31.191795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f11316-7670-46da-83b0-d9644a0952d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:31.192072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f11316-7670-46da-83b0-d9644a0952d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.193168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f11316-7670-46da-83b0-d9644a0952d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:31.196405 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f11316-7670-46da-83b0-d9644a0952d5 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:31.196965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee75d56-daef-48c3-bc81-c39409f7bb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:31.197996 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee75d56-daef-48c3-bc81-c39409f7bb6c admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:31.198540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee75d56-daef-48c3-bc81-c39409f7bb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:31.199076 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee75d56-daef-48c3-bc81-c39409f7bb6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.201231 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee75d56-daef-48c3-bc81-c39409f7bb6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:31.204831 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee75d56-daef-48c3-bc81-c39409f7bb6c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:31.224041 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 795/1569] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:31.225091 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee75d56-daef-48c3-bc81-c39409f7bb6c admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:31.238579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee75d56-daef-48c3-bc81-c39409f7bb6c admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:31.244028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b83b9b-ebaa-47e3-9d04-e17c0431a190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:31.244701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b83b9b-ebaa-47e3-9d04-e17c0431a190 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:31.244990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b83b9b-ebaa-47e3-9d04-e17c0431a190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:31.245269 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b83b9b-ebaa-47e3-9d04-e17c0431a190 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.246184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b83b9b-ebaa-47e3-9d04-e17c0431a190 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:31.250002 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b83b9b-ebaa-47e3-9d04-e17c0431a190 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:31.250277 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 775/1570] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:12:30 2022] POST /identity/v3/projects => generated 377 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:31.257418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f11316-7670-46da-83b0-d9644a0952d5 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:31.298982 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 796/1571] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:12:31 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:31.300546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b83b9b-ebaa-47e3-9d04-e17c0431a190 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:31.372197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f11316-7670-46da-83b0-d9644a0952d5 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:31.427235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b83b9b-ebaa-47e3-9d04-e17c0431a190 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:31.435419 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030facef-ee0f-4bcd-a781-d6abf0b09f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:31.436526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030facef-ee0f-4bcd-a781-d6abf0b09f8b admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:31.437158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030facef-ee0f-4bcd-a781-d6abf0b09f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:31.437649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030facef-ee0f-4bcd-a781-d6abf0b09f8b admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.438902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030facef-ee0f-4bcd-a781-d6abf0b09f8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:31.441593 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:31.442626 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030facef-ee0f-4bcd-a781-d6abf0b09f8b admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:31.468599 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de5ec2-293e-4b3c-924d-8a08e6b175c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:31.469666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de5ec2-293e-4b3c-924d-8a08e6b175c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:31.470076 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de5ec2-293e-4b3c-924d-8a08e6b175c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:31.471378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de5ec2-293e-4b3c-924d-8a08e6b175c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.471822 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de5ec2-293e-4b3c-924d-8a08e6b175c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:31.472174 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de5ec2-293e-4b3c-924d-8a08e6b175c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:31.475078 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de5ec2-293e-4b3c-924d-8a08e6b175c1 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:31.480337 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 797/1572] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:12:31 2022] POST /identity/v3/users => generated 302 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:31.481422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030facef-ee0f-4bcd-a781-d6abf0b09f8b admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:31.493430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 776/1573] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:31.494835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de5ec2-293e-4b3c-924d-8a08e6b175c1 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:31.503084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030facef-ee0f-4bcd-a781-d6abf0b09f8b admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:31.515634 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a06f2-308b-4923-9e52-7c3d4c8d21d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:31.516096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a06f2-308b-4923-9e52-7c3d4c8d21d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:31.516438 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a06f2-308b-4923-9e52-7c3d4c8d21d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:31.516738 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de5ec2-293e-4b3c-924d-8a08e6b175c1 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:31.517788 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a06f2-308b-4923-9e52-7c3d4c8d21d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.518367 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a06f2-308b-4923-9e52-7c3d4c8d21d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:31.521806 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a06f2-308b-4923-9e52-7c3d4c8d21d9 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:31.523016 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ca26f-3cba-4de8-a481-3857f66152e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:31.523851 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ca26f-3cba-4de8-a481-3857f66152e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:31.524219 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ca26f-3cba-4de8-a481-3857f66152e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:31.524950 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ca26f-3cba-4de8-a481-3857f66152e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.526177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ca26f-3cba-4de8-a481-3857f66152e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:31.526726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ca26f-3cba-4de8-a481-3857f66152e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:31.529198 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ca26f-3cba-4de8-a481-3857f66152e6 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:31.550974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 777/1574] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:31.552160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 798/1575] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:12:31 2022] POST /identity/v3/users => generated 295 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:31.552668 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ca26f-3cba-4de8-a481-3857f66152e6 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:31.554844 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a06f2-308b-4923-9e52-7c3d4c8d21d9 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:31.571437 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ca26f-3cba-4de8-a481-3857f66152e6 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:31.573403 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a06f2-308b-4923-9e52-7c3d4c8d21d9 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:31.577354 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b5725-ab9b-47bb-b8de-7a6e7b6b460f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:31.578126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b5725-ab9b-47bb-b8de-7a6e7b6b460f admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:31.578326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b5725-ab9b-47bb-b8de-7a6e7b6b460f admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:31.578569 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b5725-ab9b-47bb-b8de-7a6e7b6b460f admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.579231 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b5725-ab9b-47bb-b8de-7a6e7b6b460f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:31.579515 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77924a-a5b5-4579-9b1c-e3a27f9db915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:31.580402 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77924a-a5b5-4579-9b1c-e3a27f9db915 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:31.580843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77924a-a5b5-4579-9b1c-e3a27f9db915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:31.581084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b5725-ab9b-47bb-b8de-7a6e7b6b460f admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:31.581330 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77924a-a5b5-4579-9b1c-e3a27f9db915 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.582132 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77924a-a5b5-4579-9b1c-e3a27f9db915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:31.584376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77924a-a5b5-4579-9b1c-e3a27f9db915 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:31.598600 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 778/1576] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:31.598937 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 799/1577] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:31.604787 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77924a-a5b5-4579-9b1c-e3a27f9db915 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:31.605097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b5725-ab9b-47bb-b8de-7a6e7b6b460f admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:31.619400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77924a-a5b5-4579-9b1c-e3a27f9db915 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:31.620422 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b5725-ab9b-47bb-b8de-7a6e7b6b460f admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:31.625804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41febe-2af1-4a2b-bbc9-dd774ffc3af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:31.626426 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41febe-2af1-4a2b-bbc9-dd774ffc3af8 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:31.626708 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41febe-2af1-4a2b-bbc9-dd774ffc3af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:31.626992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895a271-ed25-4993-9018-a67c487ec903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:31.627244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41febe-2af1-4a2b-bbc9-dd774ffc3af8 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.627499 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895a271-ed25-4993-9018-a67c487ec903 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:31.627684 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41febe-2af1-4a2b-bbc9-dd774ffc3af8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:31.628224 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895a271-ed25-4993-9018-a67c487ec903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:31.628722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895a271-ed25-4993-9018-a67c487ec903 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.629898 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4895a271-ed25-4993-9018-a67c487ec903 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:31.630074 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41febe-2af1-4a2b-bbc9-dd774ffc3af8 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:31.632297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4895a271-ed25-4993-9018-a67c487ec903 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:31.664619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 800/1578] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:12:31 2022] POST /identity/v3/users => generated 287 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:31.665856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895a271-ed25-4993-9018-a67c487ec903 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:31.674848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 779/1579] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:12:31 2022] POST /identity/v3/users => generated 289 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:31.676475 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41febe-2af1-4a2b-bbc9-dd774ffc3af8 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:31.677766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895a271-ed25-4993-9018-a67c487ec903 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:31.684281 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238e9aa-aaa8-4c75-b040-48be01e4217f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:31.685167 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238e9aa-aaa8-4c75-b040-48be01e4217f admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:31.685475 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238e9aa-aaa8-4c75-b040-48be01e4217f admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:31.685812 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238e9aa-aaa8-4c75-b040-48be01e4217f admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.686621 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238e9aa-aaa8-4c75-b040-48be01e4217f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:31.688622 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238e9aa-aaa8-4c75-b040-48be01e4217f admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:31.691243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41febe-2af1-4a2b-bbc9-dd774ffc3af8 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:31.696671 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd764de-781a-44b1-b4f9-8f82e3dee861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:31.697453 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd764de-781a-44b1-b4f9-8f82e3dee861 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:31.697818 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd764de-781a-44b1-b4f9-8f82e3dee861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:31.698164 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd764de-781a-44b1-b4f9-8f82e3dee861 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.698924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd764de-781a-44b1-b4f9-8f82e3dee861 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:31.701614 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd764de-781a-44b1-b4f9-8f82e3dee861 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:31.702380 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 801/1580] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:31.703272 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238e9aa-aaa8-4c75-b040-48be01e4217f admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:31.717765 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238e9aa-aaa8-4c75-b040-48be01e4217f admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:31.726050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc6973-9279-484f-879d-e8c670bc9282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:31.727119 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc6973-9279-484f-879d-e8c670bc9282 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:31.727624 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc6973-9279-484f-879d-e8c670bc9282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:31.728156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc6973-9279-484f-879d-e8c670bc9282 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.729550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc6973-9279-484f-879d-e8c670bc9282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:31.733129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc6973-9279-484f-879d-e8c670bc9282 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:31.738633 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:31.738968 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 780/1581] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:12:31 2022] POST /identity/v3/projects => generated 391 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:31.740873 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd764de-781a-44b1-b4f9-8f82e3dee861 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:31.757334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 802/1582] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:31.758930 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc6973-9279-484f-879d-e8c670bc9282 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:31.773270 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc6973-9279-484f-879d-e8c670bc9282 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:31.860882 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38319bd-62d9-46d5-98e6-1326304c3eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:31.861989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38319bd-62d9-46d5-98e6-1326304c3eb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:31.862359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38319bd-62d9-46d5-98e6-1326304c3eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:31.862753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38319bd-62d9-46d5-98e6-1326304c3eb6 admin admin] PATH_INFO: `/v3/projects/a6ed649585c7475a96ee1eb5ebd60eba/users/0e52cf55d1ac4f37929f4895feba167e/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.878586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd764de-781a-44b1-b4f9-8f82e3dee861 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:31.882464 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38319bd-62d9-46d5-98e6-1326304c3eb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ed649585c7475a96ee1eb5ebd60eba, user_id=0e52cf55d1ac4f37929f4895feba167e, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:31.883703 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: WARNING py.warnings [None req-b38319bd-62d9-46d5-98e6-1326304c3eb6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:12:31.883703 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: warnings.warn(msg) Aug 01 15:12:31.883703 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: Aug 01 15:12:31.886533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38319bd-62d9-46d5-98e6-1326304c3eb6 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:31.887684 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d069ea-c403-4973-96a5-4081aac95310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:31.888555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d069ea-c403-4973-96a5-4081aac95310 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:31.888973 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d069ea-c403-4973-96a5-4081aac95310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:31.889459 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d069ea-c403-4973-96a5-4081aac95310 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.890493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d069ea-c403-4973-96a5-4081aac95310 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:31.894303 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d069ea-c403-4973-96a5-4081aac95310 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:31.909745 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 803/1583] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:31 2022] PUT /identity/v3/projects/a6ed649585c7475a96ee1eb5ebd60eba/users/0e52cf55d1ac4f37929f4895feba167e/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:31.911078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38319bd-62d9-46d5-98e6-1326304c3eb6 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:31.942199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 781/1584] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:12:31 2022] POST /identity/v3/projects => generated 369 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:31.943976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d069ea-c403-4973-96a5-4081aac95310 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:32.026200 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38319bd-62d9-46d5-98e6-1326304c3eb6 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:32.060030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d069ea-c403-4973-96a5-4081aac95310 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:32.071109 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b06b4d-4f32-4258-9d96-8ed10107683b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:32.072983 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b06b4d-4f32-4258-9d96-8ed10107683b admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.073321 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b06b4d-4f32-4258-9d96-8ed10107683b admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.073827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b06b4d-4f32-4258-9d96-8ed10107683b admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.075565 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b06b4d-4f32-4258-9d96-8ed10107683b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:32.078279 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b06b4d-4f32-4258-9d96-8ed10107683b admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:32.080119 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:32.101068 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 782/1585] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:32.103587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b06b4d-4f32-4258-9d96-8ed10107683b admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:32.116675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b06b4d-4f32-4258-9d96-8ed10107683b admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:32.119775 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95196677-c4e5-4775-8914-a5b36c1c8ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:32.120674 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95196677-c4e5-4775-8914-a5b36c1c8ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.121115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95196677-c4e5-4775-8914-a5b36c1c8ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.121426 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95196677-c4e5-4775-8914-a5b36c1c8ac5 admin admin] PATH_INFO: `/v3/projects/916870664da8419eb5466fd75b15987d/users/5a49850f6d8a474d83821f5f49dff06c/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.122354 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa2007-bceb-4a7b-b272-fa236a744a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:32.123099 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa2007-bceb-4a7b-b272-fa236a744a11 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.123521 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa2007-bceb-4a7b-b272-fa236a744a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.123931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa2007-bceb-4a7b-b272-fa236a744a11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.125127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa2007-bceb-4a7b-b272-fa236a744a11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:32.128349 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa2007-bceb-4a7b-b272-fa236a744a11 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:32.141790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95196677-c4e5-4775-8914-a5b36c1c8ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=916870664da8419eb5466fd75b15987d, user_id=5a49850f6d8a474d83821f5f49dff06c, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:32.146155 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 783/1586] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:32.146155 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa2007-bceb-4a7b-b272-fa236a744a11 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:32.146410 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95196677-c4e5-4775-8914-a5b36c1c8ac5 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:32.163988 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa2007-bceb-4a7b-b272-fa236a744a11 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:32.165072 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 804/1587] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:31 2022] PUT /identity/v3/projects/916870664da8419eb5466fd75b15987d/users/5a49850f6d8a474d83821f5f49dff06c/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:32.165969 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95196677-c4e5-4775-8914-a5b36c1c8ac5 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:32.187840 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:32.276029 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d927329-3677-48eb-802d-267755c4d9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:32.277094 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d927329-3677-48eb-802d-267755c4d9c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.278119 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d927329-3677-48eb-802d-267755c4d9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.278599 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d927329-3677-48eb-802d-267755c4d9c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.279912 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d927329-3677-48eb-802d-267755c4d9c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:32.283166 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d927329-3677-48eb-802d-267755c4d9c4 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:32.288745 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95196677-c4e5-4775-8914-a5b36c1c8ac5 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:32.298048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a043b-7692-4b3e-919b-4c646dd50522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:32.298369 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a043b-7692-4b3e-919b-4c646dd50522 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.298729 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a043b-7692-4b3e-919b-4c646dd50522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.299220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a043b-7692-4b3e-919b-4c646dd50522 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.300510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a043b-7692-4b3e-919b-4c646dd50522 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:32.304078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a043b-7692-4b3e-919b-4c646dd50522 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:32.309685 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 784/1588] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:32.309685 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d927329-3677-48eb-802d-267755c4d9c4 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:32.325701 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d927329-3677-48eb-802d-267755c4d9c4 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:32.333285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f0d5a-a1ec-438d-8674-330b1e333fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:32.333895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f0d5a-a1ec-438d-8674-330b1e333fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.334170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f0d5a-a1ec-438d-8674-330b1e333fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.334617 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f0d5a-a1ec-438d-8674-330b1e333fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.335332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f0d5a-a1ec-438d-8674-330b1e333fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:32.337117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f0d5a-a1ec-438d-8674-330b1e333fc8 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:32.352069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 805/1589] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:12:32 2022] POST /identity/v3/users => generated 296 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:32.352069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c887d9-e60c-461c-8f18-c33b98a826c9 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.352069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c887d9-e60c-461c-8f18-c33b98a826c9 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.352069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c887d9-e60c-461c-8f18-c33b98a826c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.353263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 785/1590] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:32.353263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f0d5a-a1ec-438d-8674-330b1e333fc8 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:32.365133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f0d5a-a1ec-438d-8674-330b1e333fc8 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:32.371715 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f0ed3-f127-4248-a623-d28ff3858e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:32.372390 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f0ed3-f127-4248-a623-d28ff3858e1d admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.372737 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f0ed3-f127-4248-a623-d28ff3858e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.373048 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f0ed3-f127-4248-a623-d28ff3858e1d admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.373726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f0ed3-f127-4248-a623-d28ff3858e1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:32.375703 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f0ed3-f127-4248-a623-d28ff3858e1d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:32.399237 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-25c887d9-e60c-461c-8f18-c33b98a826c9 None None] MFA Rules not processed for user `0e52cf55d1ac4f37929f4895feba167e`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:32.401149 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-25c887d9-e60c-461c-8f18-c33b98a826c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:32.409849 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 786/1591] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:12:32 2022] POST /identity/v3/users => generated 285 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:32.410767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f0ed3-f127-4248-a623-d28ff3858e1d admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:32.424761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f0ed3-f127-4248-a623-d28ff3858e1d admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:32.430981 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b2abb-8b28-4363-b6dc-afe5b57242c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:32.432127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b2abb-8b28-4363-b6dc-afe5b57242c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.432560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b2abb-8b28-4363-b6dc-afe5b57242c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.432845 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b2abb-8b28-4363-b6dc-afe5b57242c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.433642 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b2abb-8b28-4363-b6dc-afe5b57242c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:32.435352 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b2abb-8b28-4363-b6dc-afe5b57242c8 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:32.448716 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 787/1592] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:32.450208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b2abb-8b28-4363-b6dc-afe5b57242c8 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:32.463541 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b2abb-8b28-4363-b6dc-afe5b57242c8 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:32.469645 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead4f8d-8174-4a17-be08-2b6e0a5a6858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:32.470177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead4f8d-8174-4a17-be08-2b6e0a5a6858 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.470457 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead4f8d-8174-4a17-be08-2b6e0a5a6858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.470729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead4f8d-8174-4a17-be08-2b6e0a5a6858 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.471435 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead4f8d-8174-4a17-be08-2b6e0a5a6858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:32.473133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead4f8d-8174-4a17-be08-2b6e0a5a6858 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:32.486548 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 788/1593] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:32.488070 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f5dc0-792e-4074-a8ba-f2357b2893d3 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.488570 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f5dc0-792e-4074-a8ba-f2357b2893d3 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.488977 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f5dc0-792e-4074-a8ba-f2357b2893d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.512965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-db7f5dc0-792e-4074-a8ba-f2357b2893d3 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:32.514260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-db7f5dc0-792e-4074-a8ba-f2357b2893d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:32.514436 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 806/1594] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:32 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:32.515506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c887d9-e60c-461c-8f18-c33b98a826c9 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:32.528885 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 789/1595] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:32.529994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f5dc0-792e-4074-a8ba-f2357b2893d3 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:32.533882 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c887d9-e60c-461c-8f18-c33b98a826c9 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:32.540678 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618acdde-d0ad-4e4c-8470-22562e702706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:32.541659 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618acdde-d0ad-4e4c-8470-22562e702706 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.542074 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618acdde-d0ad-4e4c-8470-22562e702706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.542244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f5dc0-792e-4074-a8ba-f2357b2893d3 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:32.542565 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618acdde-d0ad-4e4c-8470-22562e702706 admin admin] PATH_INFO: `/v3/projects/78a248f087db456aa6e7464390879d86/users/4d5cd12c706e41918f88abdcefc4926b/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.550097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7cacf8-32d1-4e57-9caa-bf8b57767e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:32.551090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cacf8-32d1-4e57-9caa-bf8b57767e74 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.551532 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cacf8-32d1-4e57-9caa-bf8b57767e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.551958 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cacf8-32d1-4e57-9caa-bf8b57767e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.553220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7cacf8-32d1-4e57-9caa-bf8b57767e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:32.556081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7cacf8-32d1-4e57-9caa-bf8b57767e74 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:32.569173 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618acdde-d0ad-4e4c-8470-22562e702706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78a248f087db456aa6e7464390879d86, user_id=4d5cd12c706e41918f88abdcefc4926b, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:32.574283 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618acdde-d0ad-4e4c-8470-22562e702706 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:32.575256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 790/1596] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:32.576373 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7cacf8-32d1-4e57-9caa-bf8b57767e74 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:32.599677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7cacf8-32d1-4e57-9caa-bf8b57767e74 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:32.601850 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 807/1597] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:32 2022] PUT /identity/v3/projects/78a248f087db456aa6e7464390879d86/users/4d5cd12c706e41918f88abdcefc4926b/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:32.605632 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618acdde-d0ad-4e4c-8470-22562e702706 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:32.691639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618acdde-d0ad-4e4c-8470-22562e702706 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:32.699914 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb94021-463c-4df6-8c37-a42412e5edea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:32.700849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb94021-463c-4df6-8c37-a42412e5edea admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.701195 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb94021-463c-4df6-8c37-a42412e5edea admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.701561 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb94021-463c-4df6-8c37-a42412e5edea admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.702554 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb94021-463c-4df6-8c37-a42412e5edea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:32.705217 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb94021-463c-4df6-8c37-a42412e5edea admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:32.706127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd09ae-9805-472f-a4c7-cbb932981bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:32.706839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd09ae-9805-472f-a4c7-cbb932981bcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.707128 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd09ae-9805-472f-a4c7-cbb932981bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.707427 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd09ae-9805-472f-a4c7-cbb932981bcb admin admin] PATH_INFO: `/v3/projects/8b18759847b44a698a092e4484b244d0/users/3c18437883ce42369c7563224c208523/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.720777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 808/1598] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:32.721630 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb94021-463c-4df6-8c37-a42412e5edea admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:32.727834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd09ae-9805-472f-a4c7-cbb932981bcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b18759847b44a698a092e4484b244d0, user_id=3c18437883ce42369c7563224c208523, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:32.728062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:32.734507 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd09ae-9805-472f-a4c7-cbb932981bcb admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:32.740090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb94021-463c-4df6-8c37-a42412e5edea admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:32.741011 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:32.752499 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd569221-7545-4566-866e-2d578df986f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:32.753656 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd569221-7545-4566-866e-2d578df986f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.754156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd569221-7545-4566-866e-2d578df986f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.754650 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd569221-7545-4566-866e-2d578df986f8 admin admin] PATH_INFO: `/v3/projects/150274fc041d4b60b1d44e2a8b70000a/users/4629e740eaa34b009fd68de2df7814c4/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.759075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 791/1599] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:32 2022] PUT /identity/v3/projects/8b18759847b44a698a092e4484b244d0/users/3c18437883ce42369c7563224c208523/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:32.760275 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd09ae-9805-472f-a4c7-cbb932981bcb admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:32.784380 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd569221-7545-4566-866e-2d578df986f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=150274fc041d4b60b1d44e2a8b70000a, user_id=4629e740eaa34b009fd68de2df7814c4, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:32.790192 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd569221-7545-4566-866e-2d578df986f8 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:32.817898 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 809/1600] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:32 2022] PUT /identity/v3/projects/150274fc041d4b60b1d44e2a8b70000a/users/4629e740eaa34b009fd68de2df7814c4/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:32.819819 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd569221-7545-4566-866e-2d578df986f8 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:32.886417 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd09ae-9805-472f-a4c7-cbb932981bcb admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:32.943277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd569221-7545-4566-866e-2d578df986f8 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:32.943277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56238de-ba6a-4b5b-9587-86f746817159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:32.943277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56238de-ba6a-4b5b-9587-86f746817159 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.944822 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56238de-ba6a-4b5b-9587-86f746817159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.944822 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56238de-ba6a-4b5b-9587-86f746817159 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.945430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56238de-ba6a-4b5b-9587-86f746817159 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:32.948098 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56238de-ba6a-4b5b-9587-86f746817159 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:32.993698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 810/1601] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:12:32 2022] POST /identity/v3/projects => generated 367 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:32.993942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb943e-124e-4777-ace4-2804a3747dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:32.994876 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56238de-ba6a-4b5b-9587-86f746817159 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:32.995066 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb943e-124e-4777-ace4-2804a3747dcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.995499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb943e-124e-4777-ace4-2804a3747dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.996003 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb943e-124e-4777-ace4-2804a3747dcf admin admin] PATH_INFO: `/v3/projects/51e95123daab4cd285589cb523685623/users/e708da2136a248c196ae4ecb531bb322/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.017208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb943e-124e-4777-ace4-2804a3747dcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=51e95123daab4cd285589cb523685623, user_id=e708da2136a248c196ae4ecb531bb322, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:33.019778 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb943e-124e-4777-ace4-2804a3747dcf admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:33.044541 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 792/1602] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:32 2022] PUT /identity/v3/projects/51e95123daab4cd285589cb523685623/users/e708da2136a248c196ae4ecb531bb322/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:33.046212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb943e-124e-4777-ace4-2804a3747dcf admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:33.082927 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56238de-ba6a-4b5b-9587-86f746817159 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:33.159576 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb943e-124e-4777-ace4-2804a3747dcf admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:33.171482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42594041-55c4-4367-92e3-d5999095dd03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:33.172558 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e903c-91fb-4d76-bee5-efadf6added0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:33.173015 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42594041-55c4-4367-92e3-d5999095dd03 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.173288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e903c-91fb-4d76-bee5-efadf6added0 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.173514 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42594041-55c4-4367-92e3-d5999095dd03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.173663 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e903c-91fb-4d76-bee5-efadf6added0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.173998 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42594041-55c4-4367-92e3-d5999095dd03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.174161 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e903c-91fb-4d76-bee5-efadf6added0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.174994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e903c-91fb-4d76-bee5-efadf6added0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:33.177008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e903c-91fb-4d76-bee5-efadf6added0 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:33.191604 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42594041-55c4-4367-92e3-d5999095dd03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:33.192449 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 793/1603] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:33.193648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d850e-7635-42e5-8bd4-93e79ba370b0 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.194018 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d850e-7635-42e5-8bd4-93e79ba370b0 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.194414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d850e-7635-42e5-8bd4-93e79ba370b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.194592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42594041-55c4-4367-92e3-d5999095dd03 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:33.213484 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 811/1604] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:12:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:33.214629 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42594041-55c4-4367-92e3-d5999095dd03 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:33.235195 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42594041-55c4-4367-92e3-d5999095dd03 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:33.245114 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360d3af-2a96-40ff-b7c7-d3fa4028e5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:33.245524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-978d850e-7635-42e5-8bd4-93e79ba370b0 None None] MFA Rules not processed for user `4d5cd12c706e41918f88abdcefc4926b`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:33.246034 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360d3af-2a96-40ff-b7c7-d3fa4028e5ae admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.246526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360d3af-2a96-40ff-b7c7-d3fa4028e5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.246831 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360d3af-2a96-40ff-b7c7-d3fa4028e5ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.247017 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-978d850e-7635-42e5-8bd4-93e79ba370b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:33.248105 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360d3af-2a96-40ff-b7c7-d3fa4028e5ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:33.251077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360d3af-2a96-40ff-b7c7-d3fa4028e5ae admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:33.261735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:33.275401 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 812/1605] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:33.275401 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577197aa-44ef-4d59-84ac-3ee97e84be0b None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.275401 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577197aa-44ef-4d59-84ac-3ee97e84be0b None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.275401 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577197aa-44ef-4d59-84ac-3ee97e84be0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.312747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-577197aa-44ef-4d59-84ac-3ee97e84be0b None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:33.314954 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-577197aa-44ef-4d59-84ac-3ee97e84be0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:33.331260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 813/1606] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:33.334203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ddb5f8-feb4-4579-ab08-da308b402d78 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.334701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ddb5f8-feb4-4579-ab08-da308b402d78 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.335198 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ddb5f8-feb4-4579-ab08-da308b402d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.350694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:33.388396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-76ddb5f8-feb4-4579-ab08-da308b402d78 None None] MFA Rules not processed for user `4629e740eaa34b009fd68de2df7814c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:33.390162 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-76ddb5f8-feb4-4579-ab08-da308b402d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:33.401612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 794/1607] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:33.403430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d850e-7635-42e5-8bd4-93e79ba370b0 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:33.419212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d850e-7635-42e5-8bd4-93e79ba370b0 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:33.425689 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f175400-bbd5-479e-825a-2cd85b3ef60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:33.426336 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f175400-bbd5-479e-825a-2cd85b3ef60e admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.426649 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f175400-bbd5-479e-825a-2cd85b3ef60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.426918 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f175400-bbd5-479e-825a-2cd85b3ef60e admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.427605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f175400-bbd5-479e-825a-2cd85b3ef60e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:33.430704 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f175400-bbd5-479e-825a-2cd85b3ef60e admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:33.469095 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 795/1608] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:12:33 2022] POST /identity/v3/users => generated 290 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:33.472949 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2988b90-155b-44e8-8c14-1f4ba1cea764 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.472949 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2988b90-155b-44e8-8c14-1f4ba1cea764 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.473619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2988b90-155b-44e8-8c14-1f4ba1cea764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.508049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-e2988b90-155b-44e8-8c14-1f4ba1cea764 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:33.510496 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e2988b90-155b-44e8-8c14-1f4ba1cea764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:33.525680 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 814/1609] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:33.526741 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ddb5f8-feb4-4579-ab08-da308b402d78 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:33.528805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 796/1610] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:33.530377 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2988b90-155b-44e8-8c14-1f4ba1cea764 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:33.550302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ddb5f8-feb4-4579-ab08-da308b402d78 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:33.554251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2988b90-155b-44e8-8c14-1f4ba1cea764 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:33.559044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78053f3-1734-4a66-8e8e-36525c0e8c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:33.559825 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78053f3-1734-4a66-8e8e-36525c0e8c37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.560120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78053f3-1734-4a66-8e8e-36525c0e8c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.560614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78053f3-1734-4a66-8e8e-36525c0e8c37 admin admin] PATH_INFO: `/v3/projects/29b33a1fb52146918d3c0bd0d23d2dfa/users/3af3a5c1c2f94e51bee936b4762dc14c/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.561533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df49c31-952b-4b45-a687-b5109bc634f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:33.562572 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df49c31-952b-4b45-a687-b5109bc634f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.563099 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df49c31-952b-4b45-a687-b5109bc634f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.563474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df49c31-952b-4b45-a687-b5109bc634f9 admin admin] PATH_INFO: `/v3/projects/8d844e39eed04b248d2bc6a19d0f41a8/users/81e09a5f4fd74f5586f221b5680b8e4c/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.591649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78053f3-1734-4a66-8e8e-36525c0e8c37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29b33a1fb52146918d3c0bd0d23d2dfa, user_id=3af3a5c1c2f94e51bee936b4762dc14c, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:33.595940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df49c31-952b-4b45-a687-b5109bc634f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d844e39eed04b248d2bc6a19d0f41a8, user_id=81e09a5f4fd74f5586f221b5680b8e4c, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:33.597879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78053f3-1734-4a66-8e8e-36525c0e8c37 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:33.599452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df49c31-952b-4b45-a687-b5109bc634f9 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:33.619191 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 815/1611] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:33 2022] PUT /identity/v3/projects/29b33a1fb52146918d3c0bd0d23d2dfa/users/3af3a5c1c2f94e51bee936b4762dc14c/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:33.620536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78053f3-1734-4a66-8e8e-36525c0e8c37 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:33.622710 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-c78053f3-1734-4a66-8e8e-36525c0e8c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:33.630411 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 797/1612] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:33 2022] PUT /identity/v3/projects/8d844e39eed04b248d2bc6a19d0f41a8/users/81e09a5f4fd74f5586f221b5680b8e4c/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:33.632042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df49c31-952b-4b45-a687-b5109bc634f9 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:33.741257 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78053f3-1734-4a66-8e8e-36525c0e8c37 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:33.836948 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df49c31-952b-4b45-a687-b5109bc634f9 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:33.841605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969a197-319d-41f0-94c0-e658474ab393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:33.843139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969a197-319d-41f0-94c0-e658474ab393 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.843139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969a197-319d-41f0-94c0-e658474ab393 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.843139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969a197-319d-41f0-94c0-e658474ab393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.846995 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b06b55-7e77-4506-99b7-359bb00ee99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:33.848382 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b06b55-7e77-4506-99b7-359bb00ee99b admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.849008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b06b55-7e77-4506-99b7-359bb00ee99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.849642 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b06b55-7e77-4506-99b7-359bb00ee99b admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.851057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b06b55-7e77-4506-99b7-359bb00ee99b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:33.857182 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b06b55-7e77-4506-99b7-359bb00ee99b admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:33.858015 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969a197-319d-41f0-94c0-e658474ab393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:33.861928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969a197-319d-41f0-94c0-e658474ab393 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:33.886042 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 816/1613] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:12:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:33.888055 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969a197-319d-41f0-94c0-e658474ab393 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:33.912773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969a197-319d-41f0-94c0-e658474ab393 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:33.920686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 798/1614] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:12:33 2022] POST /identity/v3/projects => generated 405 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:33.922398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b06b55-7e77-4506-99b7-359bb00ee99b admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:33.926144 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cf061-644b-4f3c-833b-ee00ee194310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:33.927067 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cf061-644b-4f3c-833b-ee00ee194310 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.927468 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cf061-644b-4f3c-833b-ee00ee194310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.927865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cf061-644b-4f3c-833b-ee00ee194310 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.929789 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1cf061-644b-4f3c-833b-ee00ee194310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:33.933334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1cf061-644b-4f3c-833b-ee00ee194310 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:33.960030 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 817/1615] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:33.960030 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cf061-644b-4f3c-833b-ee00ee194310 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:33.978032 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cf061-644b-4f3c-833b-ee00ee194310 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:34.020934 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:34.053308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:34.087401 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b06b55-7e77-4506-99b7-359bb00ee99b admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:34.096654 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a7efb-cce0-4056-8645-77cbafbd8a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:34.097694 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a7efb-cce0-4056-8645-77cbafbd8a6c admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:34.098635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a7efb-cce0-4056-8645-77cbafbd8a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:34.099096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a7efb-cce0-4056-8645-77cbafbd8a6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:34.100114 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a7efb-cce0-4056-8645-77cbafbd8a6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:34.102615 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a7efb-cce0-4056-8645-77cbafbd8a6c admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:34.108985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8e7a7-b282-4bc9-8d30-e6e6518fbfa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:34.110341 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8e7a7-b282-4bc9-8d30-e6e6518fbfa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:34.110821 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8e7a7-b282-4bc9-8d30-e6e6518fbfa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:34.111313 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8e7a7-b282-4bc9-8d30-e6e6518fbfa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:34.131836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8e7a7-b282-4bc9-8d30-e6e6518fbfa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:34.135186 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8e7a7-b282-4bc9-8d30-e6e6518fbfa5 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:34.150069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 799/1616] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:12:33 2022] POST /identity/v3/projects => generated 373 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:34.150069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6cf23-d5b0-443a-af63-02f8a2d56d64 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:34.150069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6cf23-d5b0-443a-af63-02f8a2d56d64 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:34.150069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6cf23-d5b0-443a-af63-02f8a2d56d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:34.162469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 818/1617] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:12:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:34.164893 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bc4f8-ed30-4ae1-b30f-1f1784a3f74f None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:34.165072 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bc4f8-ed30-4ae1-b30f-1f1784a3f74f None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:34.165479 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bc4f8-ed30-4ae1-b30f-1f1784a3f74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:34.172253 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-bec6cf23-d5b0-443a-af63-02f8a2d56d64 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:34.174249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-bec6cf23-d5b0-443a-af63-02f8a2d56d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:34.223643 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-3e4bc4f8-ed30-4ae1-b30f-1f1784a3f74f None None] MFA Rules not processed for user `81e09a5f4fd74f5586f221b5680b8e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:34.225551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3e4bc4f8-ed30-4ae1-b30f-1f1784a3f74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:34.296198 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 800/1618] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:34.297356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6cf23-d5b0-443a-af63-02f8a2d56d64 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:34.314901 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6cf23-d5b0-443a-af63-02f8a2d56d64 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:34.322660 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfc8bb-89f3-4e6d-8026-d27685f818c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:34.323768 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfc8bb-89f3-4e6d-8026-d27685f818c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:34.324892 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfc8bb-89f3-4e6d-8026-d27685f818c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:34.325502 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfc8bb-89f3-4e6d-8026-d27685f818c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:34.326801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dfc8bb-89f3-4e6d-8026-d27685f818c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:34.329892 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dfc8bb-89f3-4e6d-8026-d27685f818c5 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:34.338258 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 819/1619] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:34.338258 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bc4f8-ed30-4ae1-b30f-1f1784a3f74f None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:34.361995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bc4f8-ed30-4ae1-b30f-1f1784a3f74f None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:34.368599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e535e-5f3b-4980-87c5-9518154fea76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:34.369501 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e535e-5f3b-4980-87c5-9518154fea76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:34.369665 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e535e-5f3b-4980-87c5-9518154fea76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:34.370025 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e535e-5f3b-4980-87c5-9518154fea76 admin admin] PATH_INFO: `/v3/projects/3759586c5a8c4fe4a7c2310513dac4c3/users/7da9afd0e0284cbd86f75d598d3ca0aa/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:34.384517 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 801/1620] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:12:34 2022] POST /identity/v3/users => generated 309 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:34.386497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfc8bb-89f3-4e6d-8026-d27685f818c5 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:34.404505 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e535e-5f3b-4980-87c5-9518154fea76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3759586c5a8c4fe4a7c2310513dac4c3, user_id=7da9afd0e0284cbd86f75d598d3ca0aa, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:34.407114 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfc8bb-89f3-4e6d-8026-d27685f818c5 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:34.408910 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e535e-5f3b-4980-87c5-9518154fea76 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:34.422804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551281f6-6112-42bf-8942-9a2e3af138b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:34.423871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551281f6-6112-42bf-8942-9a2e3af138b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:34.424347 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551281f6-6112-42bf-8942-9a2e3af138b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:34.425270 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551281f6-6112-42bf-8942-9a2e3af138b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:34.426658 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551281f6-6112-42bf-8942-9a2e3af138b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:34.429411 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551281f6-6112-42bf-8942-9a2e3af138b4 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:34.431405 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 820/1621] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:34 2022] PUT /identity/v3/projects/3759586c5a8c4fe4a7c2310513dac4c3/users/7da9afd0e0284cbd86f75d598d3ca0aa/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:34.432351 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e535e-5f3b-4980-87c5-9518154fea76 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:34.461489 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 802/1622] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:12:34 2022] POST /identity/v3/users => generated 293 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:34.470260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d137d5e-cbaf-42db-93b3-52a6c4126dc8 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:34.471156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d137d5e-cbaf-42db-93b3-52a6c4126dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:34.472207 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d137d5e-cbaf-42db-93b3-52a6c4126dc8 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:34.474803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 803/1623] 10.222.0.93 () {54 vars in 962 bytes} [Mon Aug 1 15:12:34 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:12:34.476222 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d137d5e-cbaf-42db-93b3-52a6c4126dc8 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:34.547841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e535e-5f3b-4980-87c5-9518154fea76 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:34.555835 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c25c7a-8c4f-42bc-ac62-c933cdd112eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:34.556702 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c25c7a-8c4f-42bc-ac62-c933cdd112eb service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:34.557031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c25c7a-8c4f-42bc-ac62-c933cdd112eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:34.557334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c25c7a-8c4f-42bc-ac62-c933cdd112eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:34.618280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d137d5e-cbaf-42db-93b3-52a6c4126dc8 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:34.629010 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed3ea1-4a02-4eb4-9075-002fe9c137d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:34.630193 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed3ea1-4a02-4eb4-9075-002fe9c137d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:34.630681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed3ea1-4a02-4eb4-9075-002fe9c137d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:34.631206 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed3ea1-4a02-4eb4-9075-002fe9c137d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:34.632844 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed3ea1-4a02-4eb4-9075-002fe9c137d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:34.636381 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed3ea1-4a02-4eb4-9075-002fe9c137d3 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:34.644728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:34.666752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 804/1624] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:34.668927 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed3ea1-4a02-4eb4-9075-002fe9c137d3 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:34.692319 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed3ea1-4a02-4eb4-9075-002fe9c137d3 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:34.713575 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c345a5-57a5-4a6f-8fd1-8e9d57799724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:34.713928 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c345a5-57a5-4a6f-8fd1-8e9d57799724 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:34.714272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c345a5-57a5-4a6f-8fd1-8e9d57799724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:34.715370 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c345a5-57a5-4a6f-8fd1-8e9d57799724 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:34.716709 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c345a5-57a5-4a6f-8fd1-8e9d57799724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:34.726425 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c345a5-57a5-4a6f-8fd1-8e9d57799724 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:34.735428 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c25c7a-8c4f-42bc-ac62-c933cdd112eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:34.740568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c25c7a-8c4f-42bc-ac62-c933cdd112eb service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:34.773480 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 805/1625] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:34.775191 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c345a5-57a5-4a6f-8fd1-8e9d57799724 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:34.783440 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 821/1626] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:12:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:34.785758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20dbea-e191-4ff4-8714-c099c2c99df6 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:34.787115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20dbea-e191-4ff4-8714-c099c2c99df6 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:34.787115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20dbea-e191-4ff4-8714-c099c2c99df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:34.814785 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c345a5-57a5-4a6f-8fd1-8e9d57799724 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:34.843491 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877a777-1626-435f-bdb0-6b930d06fea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:34.845824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877a777-1626-435f-bdb0-6b930d06fea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:34.846313 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877a777-1626-435f-bdb0-6b930d06fea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:34.846813 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877a777-1626-435f-bdb0-6b930d06fea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:34.847960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877a777-1626-435f-bdb0-6b930d06fea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:34.850655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877a777-1626-435f-bdb0-6b930d06fea0 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:34.908794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-ce20dbea-e191-4ff4-8714-c099c2c99df6 None None] MFA Rules not processed for user `3d9be74621d14c94b95ae94947e5abbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:34.910171 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ce20dbea-e191-4ff4-8714-c099c2c99df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:34.920999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 806/1627] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:34.920999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877a777-1626-435f-bdb0-6b930d06fea0 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:34.959861 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877a777-1626-435f-bdb0-6b930d06fea0 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:34.992173 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34893051-fe76-4d89-a1d5-d513f74f7d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:34.993351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34893051-fe76-4d89-a1d5-d513f74f7d5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:34.993839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34893051-fe76-4d89-a1d5-d513f74f7d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:34.994320 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34893051-fe76-4d89-a1d5-d513f74f7d5d admin admin] PATH_INFO: `/v3/projects/f87ff88993034b85a07e5bb18abf3704/users/79002244e2a34fa09e2151bb82307437/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:35.045546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:35.059343 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34893051-fe76-4d89-a1d5-d513f74f7d5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f87ff88993034b85a07e5bb18abf3704, user_id=79002244e2a34fa09e2151bb82307437, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:35.072845 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34893051-fe76-4d89-a1d5-d513f74f7d5d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:35.127729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 807/1628] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:34 2022] PUT /identity/v3/projects/f87ff88993034b85a07e5bb18abf3704/users/79002244e2a34fa09e2151bb82307437/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:35.129497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34893051-fe76-4d89-a1d5-d513f74f7d5d admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:35.169203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 822/1629] 10.222.0.93 () {58 vars in 1059 bytes} [Mon Aug 1 15:12:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:35.173454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20dbea-e191-4ff4-8714-c099c2c99df6 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:35.212647 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20dbea-e191-4ff4-8714-c099c2c99df6 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:35.347357 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34893051-fe76-4d89-a1d5-d513f74f7d5d admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:35.355573 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d45ab4-4462-4e24-a883-ff2b8c2f022d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:35.358735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d45ab4-4462-4e24-a883-ff2b8c2f022d admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:35.359130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d45ab4-4462-4e24-a883-ff2b8c2f022d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:35.360162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d45ab4-4462-4e24-a883-ff2b8c2f022d admin admin] PATH_INFO: `/v3/projects/3759586c5a8c4fe4a7c2310513dac4c3/users/7da9afd0e0284cbd86f75d598d3ca0aa/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:35.363696 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d45ab4-4462-4e24-a883-ff2b8c2f022d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3759586c5a8c4fe4a7c2310513dac4c3, user_id=7da9afd0e0284cbd86f75d598d3ca0aa, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:35.369273 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d45ab4-4462-4e24-a883-ff2b8c2f022d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:35.394224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 808/1630] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:35 2022] PUT /identity/v3/projects/3759586c5a8c4fe4a7c2310513dac4c3/users/7da9afd0e0284cbd86f75d598d3ca0aa/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:35.395236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d45ab4-4462-4e24-a883-ff2b8c2f022d admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:35.436881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d9625-c257-4e18-a06f-bb2b6398c597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:35.436881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d9625-c257-4e18-a06f-bb2b6398c597 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:35.436881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d9625-c257-4e18-a06f-bb2b6398c597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:35.436881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d9625-c257-4e18-a06f-bb2b6398c597 admin admin] PATH_INFO: `/v3/projects/2e65cdc3641841ba86f30f6b2bf561a1/users/2e019408968f4815b1f1dbac035755d5/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:35.500734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758d9625-c257-4e18-a06f-bb2b6398c597 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e65cdc3641841ba86f30f6b2bf561a1, user_id=2e019408968f4815b1f1dbac035755d5, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:35.523978 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758d9625-c257-4e18-a06f-bb2b6398c597 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:35.562442 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 823/1631] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:35 2022] PUT /identity/v3/projects/2e65cdc3641841ba86f30f6b2bf561a1/users/2e019408968f4815b1f1dbac035755d5/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:35.564378 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d9625-c257-4e18-a06f-bb2b6398c597 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:35.624776 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d45ab4-4462-4e24-a883-ff2b8c2f022d admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:35.655414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:35.708593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d9625-c257-4e18-a06f-bb2b6398c597 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:35.716185 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc8534-c37e-43a1-a904-4024645de5d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:35.717063 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc8534-c37e-43a1-a904-4024645de5d4 service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:35.717447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc8534-c37e-43a1-a904-4024645de5d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:35.719112 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc8534-c37e-43a1-a904-4024645de5d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:35.827679 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc8534-c37e-43a1-a904-4024645de5d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:35.829475 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc8534-c37e-43a1-a904-4024645de5d4 service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:35.843829 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 824/1632] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:12:35 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:35.859415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ce667-c332-4318-8360-59ce7ee7f463 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:35.859761 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ce667-c332-4318-8360-59ce7ee7f463 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:35.860488 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ce667-c332-4318-8360-59ce7ee7f463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:35.913716 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3f442-3606-42f2-b8d6-9d327daea90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:35.916940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3f442-3606-42f2-b8d6-9d327daea90a admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:35.922805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3f442-3606-42f2-b8d6-9d327daea90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:35.925255 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3f442-3606-42f2-b8d6-9d327daea90a admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:35.928433 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3f442-3606-42f2-b8d6-9d327daea90a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:35.935670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3f442-3606-42f2-b8d6-9d327daea90a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:35.937015 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-176ce667-c332-4318-8360-59ce7ee7f463 None None] MFA Rules not processed for user `7da9afd0e0284cbd86f75d598d3ca0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:35.939663 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-176ce667-c332-4318-8360-59ce7ee7f463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:35.976871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 809/1633] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 582 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:35.978240 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3f442-3606-42f2-b8d6-9d327daea90a admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:36.003166 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3f442-3606-42f2-b8d6-9d327daea90a admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:36.017806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c167b7-8785-4c4b-8fe8-9af979045754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:36.018840 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c167b7-8785-4c4b-8fe8-9af979045754 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:36.019279 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c167b7-8785-4c4b-8fe8-9af979045754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:36.019874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c167b7-8785-4c4b-8fe8-9af979045754 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:36.020881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c167b7-8785-4c4b-8fe8-9af979045754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:36.022979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c167b7-8785-4c4b-8fe8-9af979045754 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:36.052891 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 810/1634] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:36.054371 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c167b7-8785-4c4b-8fe8-9af979045754 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:36.084580 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c167b7-8785-4c4b-8fe8-9af979045754 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:36.101080 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efcd89b-84c7-4585-8d95-91730e8ef1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:36.102099 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcd89b-84c7-4585-8d95-91730e8ef1b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:36.102579 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcd89b-84c7-4585-8d95-91730e8ef1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:36.103011 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcd89b-84c7-4585-8d95-91730e8ef1b2 admin admin] PATH_INFO: `/v3/projects/f87ff88993034b85a07e5bb18abf3704/users/79002244e2a34fa09e2151bb82307437/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:36.105557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efcd89b-84c7-4585-8d95-91730e8ef1b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f87ff88993034b85a07e5bb18abf3704, user_id=79002244e2a34fa09e2151bb82307437, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:36.108642 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efcd89b-84c7-4585-8d95-91730e8ef1b2 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:36.163033 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 811/1635] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:36 2022] PUT /identity/v3/projects/f87ff88993034b85a07e5bb18abf3704/users/79002244e2a34fa09e2151bb82307437/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:36.165116 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efcd89b-84c7-4585-8d95-91730e8ef1b2 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:36.183067 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:36.194799 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 825/1636] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:35 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:36.197204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb056c75-7766-4fc7-904b-9df06cee35da None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:36.198294 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb056c75-7766-4fc7-904b-9df06cee35da None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:36.198294 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb056c75-7766-4fc7-904b-9df06cee35da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:36.269804 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-fb056c75-7766-4fc7-904b-9df06cee35da None None] MFA Rules not processed for user `79002244e2a34fa09e2151bb82307437`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:36.273923 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-fb056c75-7766-4fc7-904b-9df06cee35da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:36.413759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:36.442996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efcd89b-84c7-4585-8d95-91730e8ef1b2 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:36.461334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28652185-04d7-49d0-9bab-ec1636937d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:36.464144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28652185-04d7-49d0-9bab-ec1636937d74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:36.464925 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28652185-04d7-49d0-9bab-ec1636937d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:36.472264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 826/1637] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:36 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:36.474188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28652185-04d7-49d0-9bab-ec1636937d74 admin admin] PATH_INFO: `/v3/projects/2e65cdc3641841ba86f30f6b2bf561a1/users/2e019408968f4815b1f1dbac035755d5/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:36.474415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb056c75-7766-4fc7-904b-9df06cee35da None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:36.474595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28652185-04d7-49d0-9bab-ec1636937d74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e65cdc3641841ba86f30f6b2bf561a1, user_id=2e019408968f4815b1f1dbac035755d5, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:36.480962 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28652185-04d7-49d0-9bab-ec1636937d74 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:36.534900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 812/1638] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:36 2022] PUT /identity/v3/projects/2e65cdc3641841ba86f30f6b2bf561a1/users/2e019408968f4815b1f1dbac035755d5/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:36.536709 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28652185-04d7-49d0-9bab-ec1636937d74 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:36.658442 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb056c75-7766-4fc7-904b-9df06cee35da None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:36.684049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28652185-04d7-49d0-9bab-ec1636937d74 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:36.691150 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0a49a-efb7-4b08-a04d-89d2a6a7768b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:36.692311 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0a49a-efb7-4b08-a04d-89d2a6a7768b service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:36.692902 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0a49a-efb7-4b08-a04d-89d2a6a7768b service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:36.694954 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0a49a-efb7-4b08-a04d-89d2a6a7768b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:36.797410 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c29dab-fdb9-497e-9e0e-f014dc5b3cc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:36.798265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c29dab-fdb9-497e-9e0e-f014dc5b3cc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:36.798552 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c29dab-fdb9-497e-9e0e-f014dc5b3cc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:36.798911 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c29dab-fdb9-497e-9e0e-f014dc5b3cc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:36.853116 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0a49a-efb7-4b08-a04d-89d2a6a7768b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:36.856991 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0a49a-efb7-4b08-a04d-89d2a6a7768b service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:36.879805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 813/1639] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:12:36 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:36.883323 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b557c-7a0f-4d6f-96cb-bd95a1d61765 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:36.883974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b557c-7a0f-4d6f-96cb-bd95a1d61765 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:36.884564 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b557c-7a0f-4d6f-96cb-bd95a1d61765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:36.946785 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-262b557c-7a0f-4d6f-96cb-bd95a1d61765 None None] MFA Rules not processed for user `2e019408968f4815b1f1dbac035755d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:36.949622 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-262b557c-7a0f-4d6f-96cb-bd95a1d61765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:37.020500 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c29dab-fdb9-497e-9e0e-f014dc5b3cc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:37.025435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c29dab-fdb9-497e-9e0e-f014dc5b3cc5 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:37.036233 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:37.041339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 827/1640] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:12:36 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:37.044141 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6869141-a9c0-44c2-aaa8-74e8c499fd77 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:37.044438 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6869141-a9c0-44c2-aaa8-74e8c499fd77 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:37.045714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6869141-a9c0-44c2-aaa8-74e8c499fd77 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:37.057069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 828/1641] 10.222.0.93 () {54 vars in 912 bytes} [Mon Aug 1 15:12:37 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:12:37.059104 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5993d24-887f-46ca-b3d0-41849dc0d2f5 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:37.059445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5993d24-887f-46ca-b3d0-41849dc0d2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:37.059754 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5993d24-887f-46ca-b3d0-41849dc0d2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:37.090917 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-c5993d24-887f-46ca-b3d0-41849dc0d2f5 None None] MFA Rules not processed for user `3d9be74621d14c94b95ae94947e5abbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:37.092537 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-c5993d24-887f-46ca-b3d0-41849dc0d2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:37.210687 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 814/1642] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:37.213715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:37.319339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 829/1643] 10.222.0.93 () {58 vars in 1009 bytes} [Mon Aug 1 15:12:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:37.338300 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b557c-7a0f-4d6f-96cb-bd95a1d61765 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:37.367931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b557c-7a0f-4d6f-96cb-bd95a1d61765 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:37.377952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29560e4-a543-46d9-bf46-d3ea4d96dd0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:37.380082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29560e4-a543-46d9-bf46-d3ea4d96dd0e service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:37.381195 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29560e4-a543-46d9-bf46-d3ea4d96dd0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:37.381911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29560e4-a543-46d9-bf46-d3ea4d96dd0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:37.420160 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29560e4-a543-46d9-bf46-d3ea4d96dd0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:37.424147 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29560e4-a543-46d9-bf46-d3ea4d96dd0e service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:37.462706 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 815/1644] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:12:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:37.467039 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5993d24-887f-46ca-b3d0-41849dc0d2f5 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:37.517591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5993d24-887f-46ca-b3d0-41849dc0d2f5 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:37.530849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5cc61-7b17-41a9-83df-cdfc265ec8ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:37.532347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5cc61-7b17-41a9-83df-cdfc265ec8ae service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:37.533373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5cc61-7b17-41a9-83df-cdfc265ec8ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:37.534095 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5cc61-7b17-41a9-83df-cdfc265ec8ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:37.573493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5cc61-7b17-41a9-83df-cdfc265ec8ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:37.577142 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5cc61-7b17-41a9-83df-cdfc265ec8ae service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:37.623813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 830/1645] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:12:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:37.900734 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29560e4-a543-46d9-bf46-d3ea4d96dd0e service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:38.132754 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29560e4-a543-46d9-bf46-d3ea4d96dd0e service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:38.144782 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0381fd7-cb01-4062-9067-2613acd73189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:38.145640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0381fd7-cb01-4062-9067-2613acd73189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:38.151752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0381fd7-cb01-4062-9067-2613acd73189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:38.151752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0381fd7-cb01-4062-9067-2613acd73189 admin admin] PATH_INFO: `/v3/users/4d5cd12c706e41918f88abdcefc4926b` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:38.155067 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0381fd7-cb01-4062-9067-2613acd73189 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d5cd12c706e41918f88abdcefc4926b)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:38.157732 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0381fd7-cb01-4062-9067-2613acd73189 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:38.319117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e0381fd7-cb01-4062-9067-2613acd73189 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d5cd12c706e41918f88abdcefc4926b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:38.320598 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e0381fd7-cb01-4062-9067-2613acd73189 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d5cd12c706e41918f88abdcefc4926b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:38.350733 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e0381fd7-cb01-4062-9067-2613acd73189 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d5cd12c706e41918f88abdcefc4926b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:38.354039 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e0381fd7-cb01-4062-9067-2613acd73189 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d5cd12c706e41918f88abdcefc4926b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:38.434845 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e0381fd7-cb01-4062-9067-2613acd73189 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d5cd12c706e41918f88abdcefc4926b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:38.487947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 816/1646] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:12:37 2022] DELETE /identity/v3/users/4d5cd12c706e41918f88abdcefc4926b => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:38.494386 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9fa0e-1ca4-49c9-9be9-13dc118c674b None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:38.496857 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9fa0e-1ca4-49c9-9be9-13dc118c674b None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:38.497454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9fa0e-1ca4-49c9-9be9-13dc118c674b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:38.564560 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-66b9fa0e-1ca4-49c9-9be9-13dc118c674b None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:38.567414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-66b9fa0e-1ca4-49c9-9be9-13dc118c674b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:38.805097 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 831/1647] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:12:38.965225 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0381fd7-cb01-4062-9067-2613acd73189 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:38.969696 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e0381fd7-cb01-4062-9067-2613acd73189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:39.261622 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:39.338520 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0381fd7-cb01-4062-9067-2613acd73189 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:39.363091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32aa941-d468-4ab5-a27b-4b2bc840260e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:39.364372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32aa941-d468-4ab5-a27b-4b2bc840260e service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:39.364880 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32aa941-d468-4ab5-a27b-4b2bc840260e service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:39.365339 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32aa941-d468-4ab5-a27b-4b2bc840260e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:39.419723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32aa941-d468-4ab5-a27b-4b2bc840260e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:39.424064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32aa941-d468-4ab5-a27b-4b2bc840260e service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:39.455404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 817/1648] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:12:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:40.484038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9fa0e-1ca4-49c9-9be9-13dc118c674b None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:40.486760 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-66b9fa0e-1ca4-49c9-9be9-13dc118c674b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:40.559085 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9fa0e-1ca4-49c9-9be9-13dc118c674b None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:40.587741 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0203f-9fc6-4353-8dc8-8579a92d110e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:40.590403 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0203f-9fc6-4353-8dc8-8579a92d110e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:40.591728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0203f-9fc6-4353-8dc8-8579a92d110e admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:40.606470 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0203f-9fc6-4353-8dc8-8579a92d110e admin admin] PATH_INFO: `/v3/users/4629e740eaa34b009fd68de2df7814c4` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:40.617847 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0203f-9fc6-4353-8dc8-8579a92d110e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4629e740eaa34b009fd68de2df7814c4)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:40.630907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: WARNING py.warnings [None req-a4e0203f-9fc6-4353-8dc8-8579a92d110e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:12:40.630907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: warnings.warn(msg) Aug 01 15:12:40.630907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: Aug 01 15:12:40.638851 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0203f-9fc6-4353-8dc8-8579a92d110e admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:40.720600 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c573f9-6983-4e29-a70f-b421ca1f99c3 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:40.721428 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c573f9-6983-4e29-a70f-b421ca1f99c3 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:40.721973 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c573f9-6983-4e29-a70f-b421ca1f99c3 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:40.727018 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 818/1649] 10.222.0.93 () {54 vars in 962 bytes} [Mon Aug 1 15:12:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:12:40.800036 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350759a-ae47-4e7f-baa8-ea6cdf742084 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:40.801919 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350759a-ae47-4e7f-baa8-ea6cdf742084 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:40.802669 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350759a-ae47-4e7f-baa8-ea6cdf742084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:40.923363 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-1350759a-ae47-4e7f-baa8-ea6cdf742084 None None] MFA Rules not processed for user `3d9be74621d14c94b95ae94947e5abbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:40.928224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-1350759a-ae47-4e7f-baa8-ea6cdf742084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:41.015795 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a4e0203f-9fc6-4353-8dc8-8579a92d110e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4629e740eaa34b009fd68de2df7814c4'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:41.083260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a4e0203f-9fc6-4353-8dc8-8579a92d110e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4629e740eaa34b009fd68de2df7814c4'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:41.137352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a4e0203f-9fc6-4353-8dc8-8579a92d110e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4629e740eaa34b009fd68de2df7814c4'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:41.143927 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a4e0203f-9fc6-4353-8dc8-8579a92d110e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4629e740eaa34b009fd68de2df7814c4'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:41.149969 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a4e0203f-9fc6-4353-8dc8-8579a92d110e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4629e740eaa34b009fd68de2df7814c4'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:41.229649 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 819/1650] 10.222.0.93 () {58 vars in 1059 bytes} [Mon Aug 1 15:12:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:41.239133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a191e7-d57e-4ea3-9fba-95804260afdc None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:41.239607 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a191e7-d57e-4ea3-9fba-95804260afdc None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:41.240104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a191e7-d57e-4ea3-9fba-95804260afdc None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:41.244751 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 820/1651] 10.222.0.93 () {54 vars in 912 bytes} [Mon Aug 1 15:12:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:12:41.254905 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28863918-63ed-4241-ae87-a74bd58072c5 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:41.255379 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28863918-63ed-4241-ae87-a74bd58072c5 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:41.255871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28863918-63ed-4241-ae87-a74bd58072c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:41.347555 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 832/1652] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:12:40 2022] DELETE /identity/v3/users/4629e740eaa34b009fd68de2df7814c4 => generated 0 bytes in 863 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:41.354539 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb272eb-400b-4990-9614-fa32f2d30df1 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:41.355877 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb272eb-400b-4990-9614-fa32f2d30df1 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:41.358745 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb272eb-400b-4990-9614-fa32f2d30df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:41.374214 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-28863918-63ed-4241-ae87-a74bd58072c5 None None] MFA Rules not processed for user `3d9be74621d14c94b95ae94947e5abbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:41.378265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-28863918-63ed-4241-ae87-a74bd58072c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:41.443574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-dbb272eb-400b-4990-9614-fa32f2d30df1 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:41.450424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-dbb272eb-400b-4990-9614-fa32f2d30df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:41.638957 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:41.659547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 821/1653] 10.222.0.93 () {58 vars in 1009 bytes} [Mon Aug 1 15:12:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:41.686591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28863918-63ed-4241-ae87-a74bd58072c5 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:41.689756 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-28863918-63ed-4241-ae87-a74bd58072c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:41.728356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 833/1654] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:41.794631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb272eb-400b-4990-9614-fa32f2d30df1 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:41.797023 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-dbb272eb-400b-4990-9614-fa32f2d30df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:41.856212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28863918-63ed-4241-ae87-a74bd58072c5 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:41.863322 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d58cd1-dce5-43c4-9d99-220d3e312616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:41.864444 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d58cd1-dce5-43c4-9d99-220d3e312616 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:41.864978 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d58cd1-dce5-43c4-9d99-220d3e312616 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:41.865535 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d58cd1-dce5-43c4-9d99-220d3e312616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:41.921859 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d58cd1-dce5-43c4-9d99-220d3e312616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:41.924694 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d58cd1-dce5-43c4-9d99-220d3e312616 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:41.968942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 822/1655] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:12:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:12:42.094191 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb272eb-400b-4990-9614-fa32f2d30df1 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:42.108270 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446bdfa-791d-40c3-a2e9-ea95fd2bc6d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:42.110070 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446bdfa-791d-40c3-a2e9-ea95fd2bc6d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:42.112300 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446bdfa-791d-40c3-a2e9-ea95fd2bc6d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:42.113133 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446bdfa-791d-40c3-a2e9-ea95fd2bc6d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:42.148425 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d58cd1-dce5-43c4-9d99-220d3e312616 service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:42.161989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-f0d58cd1-dce5-43c4-9d99-220d3e312616 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:42.177488 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446bdfa-791d-40c3-a2e9-ea95fd2bc6d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:42.181366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446bdfa-791d-40c3-a2e9-ea95fd2bc6d6 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:42.211081 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 834/1656] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:12:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:42.233154 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d58cd1-dce5-43c4-9d99-220d3e312616 service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:42.285622 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a86a4-65a2-49a8-83ed-508bb6d67061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:42.286802 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a86a4-65a2-49a8-83ed-508bb6d67061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:42.287530 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a86a4-65a2-49a8-83ed-508bb6d67061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:42.288212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a86a4-65a2-49a8-83ed-508bb6d67061 admin admin] PATH_INFO: `/v3/projects/78a248f087db456aa6e7464390879d86` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:42.290916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a86a4-65a2-49a8-83ed-508bb6d67061 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78a248f087db456aa6e7464390879d86)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:42.293683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: WARNING py.warnings [None req-7f0a86a4-65a2-49a8-83ed-508bb6d67061 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:12:42.293683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: warnings.warn(msg) Aug 01 15:12:42.293683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: Aug 01 15:12:42.298493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a86a4-65a2-49a8-83ed-508bb6d67061 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:42.314806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:42.539771 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-7f0a86a4-65a2-49a8-83ed-508bb6d67061 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78a248f087db456aa6e7464390879d86', 'request_id': 'req-7f0a86a4-65a2-49a8-83ed-508bb6d67061'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:42.598579 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-7f0a86a4-65a2-49a8-83ed-508bb6d67061 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78a248f087db456aa6e7464390879d86', 'request_id': 'req-7f0a86a4-65a2-49a8-83ed-508bb6d67061'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:42.625231 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-7f0a86a4-65a2-49a8-83ed-508bb6d67061 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78a248f087db456aa6e7464390879d86', 'request_id': 'req-7f0a86a4-65a2-49a8-83ed-508bb6d67061'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:42.660768 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-7f0a86a4-65a2-49a8-83ed-508bb6d67061 admin admin] The token cache is being invalidate because project 78a248f087db456aa6e7464390879d86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:12:42.663450 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-7f0a86a4-65a2-49a8-83ed-508bb6d67061 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:42.668728 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-7f0a86a4-65a2-49a8-83ed-508bb6d67061 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:42.674052 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 823/1657] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:12:42 2022] DELETE /identity/v3/projects/78a248f087db456aa6e7464390879d86 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:12:42.681397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446bdfa-791d-40c3-a2e9-ea95fd2bc6d6 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:42.708378 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446bdfa-791d-40c3-a2e9-ea95fd2bc6d6 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:42.989998 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282b918-a911-4d04-8abf-ecdbdba1d209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:42.991202 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282b918-a911-4d04-8abf-ecdbdba1d209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:42.992511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282b918-a911-4d04-8abf-ecdbdba1d209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:42.992511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282b918-a911-4d04-8abf-ecdbdba1d209 admin admin] PATH_INFO: `/v3/users/79002244e2a34fa09e2151bb82307437` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:42.997406 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0282b918-a911-4d04-8abf-ecdbdba1d209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79002244e2a34fa09e2151bb82307437)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:43.006256 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0282b918-a911-4d04-8abf-ecdbdba1d209 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:43.318382 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-0282b918-a911-4d04-8abf-ecdbdba1d209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79002244e2a34fa09e2151bb82307437'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:43.337320 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-0282b918-a911-4d04-8abf-ecdbdba1d209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79002244e2a34fa09e2151bb82307437'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:43.357930 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-0282b918-a911-4d04-8abf-ecdbdba1d209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79002244e2a34fa09e2151bb82307437'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:43.358839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-0282b918-a911-4d04-8abf-ecdbdba1d209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79002244e2a34fa09e2151bb82307437'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:43.359499 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-0282b918-a911-4d04-8abf-ecdbdba1d209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79002244e2a34fa09e2151bb82307437'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:43.434243 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 835/1658] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:12:42 2022] DELETE /identity/v3/users/79002244e2a34fa09e2151bb82307437 => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:44.164358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a86a4-65a2-49a8-83ed-508bb6d67061 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:44.167166 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-7f0a86a4-65a2-49a8-83ed-508bb6d67061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:44.224086 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a86a4-65a2-49a8-83ed-508bb6d67061 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:44.235503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282b918-a911-4d04-8abf-ecdbdba1d209 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:44.239057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0282b918-a911-4d04-8abf-ecdbdba1d209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:44.279091 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:44.565998 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63447972-1104-43c5-b022-fa374c2d6f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:44.567502 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63447972-1104-43c5-b022-fa374c2d6f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:44.568197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63447972-1104-43c5-b022-fa374c2d6f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:44.569187 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63447972-1104-43c5-b022-fa374c2d6f90 admin admin] PATH_INFO: `/v3/users/81e09a5f4fd74f5586f221b5680b8e4c` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:44.571846 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63447972-1104-43c5-b022-fa374c2d6f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81e09a5f4fd74f5586f221b5680b8e4c)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:44.586937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63447972-1104-43c5-b022-fa374c2d6f90 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:44.604569 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282b918-a911-4d04-8abf-ecdbdba1d209 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:44.628175 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe631303-d05d-4764-a6a9-6694bfece9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:44.629579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe631303-d05d-4764-a6a9-6694bfece9af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:44.630057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe631303-d05d-4764-a6a9-6694bfece9af admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:44.630556 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe631303-d05d-4764-a6a9-6694bfece9af admin admin] PATH_INFO: `/v3/projects/150274fc041d4b60b1d44e2a8b70000a` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:44.636142 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe631303-d05d-4764-a6a9-6694bfece9af admin admin] RBAC: Authorizing `identity:delete_project(project_id=150274fc041d4b60b1d44e2a8b70000a)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:44.648173 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe631303-d05d-4764-a6a9-6694bfece9af admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:44.765284 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-63447972-1104-43c5-b022-fa374c2d6f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81e09a5f4fd74f5586f221b5680b8e4c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:44.777295 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-63447972-1104-43c5-b022-fa374c2d6f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81e09a5f4fd74f5586f221b5680b8e4c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:44.794917 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-63447972-1104-43c5-b022-fa374c2d6f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81e09a5f4fd74f5586f221b5680b8e4c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:44.796613 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-63447972-1104-43c5-b022-fa374c2d6f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81e09a5f4fd74f5586f221b5680b8e4c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:44.851548 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-63447972-1104-43c5-b022-fa374c2d6f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81e09a5f4fd74f5586f221b5680b8e4c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:44.897737 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fe631303-d05d-4764-a6a9-6694bfece9af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '150274fc041d4b60b1d44e2a8b70000a', 'request_id': 'req-fe631303-d05d-4764-a6a9-6694bfece9af'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:44.921931 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fe631303-d05d-4764-a6a9-6694bfece9af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '150274fc041d4b60b1d44e2a8b70000a', 'request_id': 'req-fe631303-d05d-4764-a6a9-6694bfece9af'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:44.923137 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 824/1659] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:12:44 2022] DELETE /identity/v3/users/81e09a5f4fd74f5586f221b5680b8e4c => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:12:44.924487 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63447972-1104-43c5-b022-fa374c2d6f90 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:44.933502 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-63447972-1104-43c5-b022-fa374c2d6f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:44.942060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fe631303-d05d-4764-a6a9-6694bfece9af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '150274fc041d4b60b1d44e2a8b70000a', 'request_id': 'req-fe631303-d05d-4764-a6a9-6694bfece9af'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:44.974883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fe631303-d05d-4764-a6a9-6694bfece9af admin admin] The token cache is being invalidate because project 150274fc041d4b60b1d44e2a8b70000a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:12:44.975399 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fe631303-d05d-4764-a6a9-6694bfece9af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:44.976925 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fe631303-d05d-4764-a6a9-6694bfece9af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:44.981099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 836/1660] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:12:44 2022] DELETE /identity/v3/projects/150274fc041d4b60b1d44e2a8b70000a => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:44.984387 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579da972-1f2f-44ab-a732-6d74e0a556da None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:44.985721 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579da972-1f2f-44ab-a732-6d74e0a556da None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:44.986469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579da972-1f2f-44ab-a732-6d74e0a556da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:44.987126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:45.044303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-579da972-1f2f-44ab-a732-6d74e0a556da None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:45.047766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-579da972-1f2f-44ab-a732-6d74e0a556da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:45.242344 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 837/1661] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:45.244643 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579da972-1f2f-44ab-a732-6d74e0a556da None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:45.248014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-579da972-1f2f-44ab-a732-6d74e0a556da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:45.278389 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63447972-1104-43c5-b022-fa374c2d6f90 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:45.288176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f939ddd-4f11-4129-a6d8-7373a2064fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:45.289863 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f939ddd-4f11-4129-a6d8-7373a2064fdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:45.290607 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f939ddd-4f11-4129-a6d8-7373a2064fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:45.291246 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f939ddd-4f11-4129-a6d8-7373a2064fdf admin admin] PATH_INFO: `/v3/users/0e52cf55d1ac4f37929f4895feba167e` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:45.292510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579da972-1f2f-44ab-a732-6d74e0a556da None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:45.306156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f939ddd-4f11-4129-a6d8-7373a2064fdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e52cf55d1ac4f37929f4895feba167e)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:45.310763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f939ddd-4f11-4129-a6d8-7373a2064fdf admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:45.323936 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72482b1-c198-4dd6-8be1-67fe2578b9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:45.325915 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72482b1-c198-4dd6-8be1-67fe2578b9f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:45.326507 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72482b1-c198-4dd6-8be1-67fe2578b9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:45.327080 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72482b1-c198-4dd6-8be1-67fe2578b9f0 admin admin] PATH_INFO: `/v3/users/2e019408968f4815b1f1dbac035755d5` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:45.332923 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72482b1-c198-4dd6-8be1-67fe2578b9f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e019408968f4815b1f1dbac035755d5)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:45.337616 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72482b1-c198-4dd6-8be1-67fe2578b9f0 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:45.410314 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9f939ddd-4f11-4129-a6d8-7373a2064fdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e52cf55d1ac4f37929f4895feba167e'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:45.411853 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9f939ddd-4f11-4129-a6d8-7373a2064fdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e52cf55d1ac4f37929f4895feba167e'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:45.422856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:45.429811 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9f939ddd-4f11-4129-a6d8-7373a2064fdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e52cf55d1ac4f37929f4895feba167e'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:45.431557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9f939ddd-4f11-4129-a6d8-7373a2064fdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e52cf55d1ac4f37929f4895feba167e'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:45.461226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9f939ddd-4f11-4129-a6d8-7373a2064fdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e52cf55d1ac4f37929f4895feba167e'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:45.472077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b72482b1-c198-4dd6-8be1-67fe2578b9f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e019408968f4815b1f1dbac035755d5'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:45.518443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b72482b1-c198-4dd6-8be1-67fe2578b9f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e019408968f4815b1f1dbac035755d5'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:45.529914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 825/1662] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:12:44 2022] DELETE /identity/v3/users/0e52cf55d1ac4f37929f4895feba167e => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:45.533040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f939ddd-4f11-4129-a6d8-7373a2064fdf admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:45.537763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-9f939ddd-4f11-4129-a6d8-7373a2064fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:45.542181 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b72482b1-c198-4dd6-8be1-67fe2578b9f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e019408968f4815b1f1dbac035755d5'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:45.544698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b72482b1-c198-4dd6-8be1-67fe2578b9f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e019408968f4815b1f1dbac035755d5'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:45.547984 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b72482b1-c198-4dd6-8be1-67fe2578b9f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e019408968f4815b1f1dbac035755d5'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:45.630993 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 838/1663] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:12:45 2022] DELETE /identity/v3/users/2e019408968f4815b1f1dbac035755d5 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:45.635257 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250776ab-a3a3-44f9-878e-88adbae3f351 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:45.636895 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250776ab-a3a3-44f9-878e-88adbae3f351 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:45.641349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250776ab-a3a3-44f9-878e-88adbae3f351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:45.721125 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-250776ab-a3a3-44f9-878e-88adbae3f351 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:45.722633 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-250776ab-a3a3-44f9-878e-88adbae3f351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:45.771035 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f939ddd-4f11-4129-a6d8-7373a2064fdf admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:45.875435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 839/1664] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:45.897299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:45.918559 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847f379-1ea0-45b3-b2a0-235d2a9d1adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:45.920217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847f379-1ea0-45b3-b2a0-235d2a9d1adf service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:45.921473 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847f379-1ea0-45b3-b2a0-235d2a9d1adf service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:45.922453 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847f379-1ea0-45b3-b2a0-235d2a9d1adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:45.937899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-9847f379-1ea0-45b3-b2a0-235d2a9d1adf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:45.971161 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9847f379-1ea0-45b3-b2a0-235d2a9d1adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:45.974192 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9847f379-1ea0-45b3-b2a0-235d2a9d1adf service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:46.000940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 826/1665] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:12:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:46.021261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250776ab-a3a3-44f9-878e-88adbae3f351 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:46.023020 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-250776ab-a3a3-44f9-878e-88adbae3f351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:46.044390 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847f379-1ea0-45b3-b2a0-235d2a9d1adf service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:46.048034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-9847f379-1ea0-45b3-b2a0-235d2a9d1adf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:46.062655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250776ab-a3a3-44f9-878e-88adbae3f351 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:46.086989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42197a1b-d375-468d-ae7a-a095fa731fab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:46.088139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42197a1b-d375-468d-ae7a-a095fa731fab service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:46.088681 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42197a1b-d375-468d-ae7a-a095fa731fab service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:46.089170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42197a1b-d375-468d-ae7a-a095fa731fab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:46.114079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847f379-1ea0-45b3-b2a0-235d2a9d1adf service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:46.129799 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42197a1b-d375-468d-ae7a-a095fa731fab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:46.135419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f7844-c84d-4849-8404-1cc94322a8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:46.136125 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f7844-c84d-4849-8404-1cc94322a8c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:46.136722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f7844-c84d-4849-8404-1cc94322a8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:46.137254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f7844-c84d-4849-8404-1cc94322a8c6 admin admin] PATH_INFO: `/v3/projects/f87ff88993034b85a07e5bb18abf3704` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:46.141863 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42197a1b-d375-468d-ae7a-a095fa731fab service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:46.147382 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f7844-c84d-4849-8404-1cc94322a8c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f87ff88993034b85a07e5bb18abf3704)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:46.151001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f7844-c84d-4849-8404-1cc94322a8c6 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:46.184163 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 840/1666] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:12:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:12:46.342306 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e57f7844-c84d-4849-8404-1cc94322a8c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f87ff88993034b85a07e5bb18abf3704', 'request_id': 'req-e57f7844-c84d-4849-8404-1cc94322a8c6'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:46.364989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e57f7844-c84d-4849-8404-1cc94322a8c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f87ff88993034b85a07e5bb18abf3704', 'request_id': 'req-e57f7844-c84d-4849-8404-1cc94322a8c6'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:46.384283 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e57f7844-c84d-4849-8404-1cc94322a8c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f87ff88993034b85a07e5bb18abf3704', 'request_id': 'req-e57f7844-c84d-4849-8404-1cc94322a8c6'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:46.436229 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e57f7844-c84d-4849-8404-1cc94322a8c6 admin admin] The token cache is being invalidate because project f87ff88993034b85a07e5bb18abf3704 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:12:46.439698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e57f7844-c84d-4849-8404-1cc94322a8c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:46.443640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e57f7844-c84d-4849-8404-1cc94322a8c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:46.453663 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 827/1667] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:12:46 2022] DELETE /identity/v3/projects/f87ff88993034b85a07e5bb18abf3704 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:46.460591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd05aa5-33bf-4454-be84-3945f37a4fd1 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:46.462260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd05aa5-33bf-4454-be84-3945f37a4fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:46.463057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd05aa5-33bf-4454-be84-3945f37a4fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:46.535432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-4dd05aa5-33bf-4454-be84-3945f37a4fd1 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:46.539948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4dd05aa5-33bf-4454-be84-3945f37a4fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:46.817013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 841/1668] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:46.829398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5887a-6074-4dd9-829c-ac16b4b76f0e None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:46.830155 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5887a-6074-4dd9-829c-ac16b4b76f0e None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:46.830671 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5887a-6074-4dd9-829c-ac16b4b76f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:46.893885 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-09c5887a-6074-4dd9-829c-ac16b4b76f0e None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:46.895595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-09c5887a-6074-4dd9-829c-ac16b4b76f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:46.919328 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 828/1669] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:46.924264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd05aa5-33bf-4454-be84-3945f37a4fd1 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:46.944101 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd05aa5-33bf-4454-be84-3945f37a4fd1 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:46.958673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7eb9e-bd8f-4afe-b4fc-74829837b5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:46.959354 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7eb9e-bd8f-4afe-b4fc-74829837b5a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:46.959671 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7eb9e-bd8f-4afe-b4fc-74829837b5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:46.960930 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7eb9e-bd8f-4afe-b4fc-74829837b5a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:46.960930 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7eb9e-bd8f-4afe-b4fc-74829837b5a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:46.961215 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7eb9e-bd8f-4afe-b4fc-74829837b5a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:46.963288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7eb9e-bd8f-4afe-b4fc-74829837b5a0 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:47.008261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 842/1670] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:47.014657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c5887a-6074-4dd9-829c-ac16b4b76f0e None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:47.035626 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c5887a-6074-4dd9-829c-ac16b4b76f0e None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:47.052972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df5aed-735f-4a04-a5c4-f30b4d3e3833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:47.053724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df5aed-735f-4a04-a5c4-f30b4d3e3833 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:47.054011 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df5aed-735f-4a04-a5c4-f30b4d3e3833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:47.054294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df5aed-735f-4a04-a5c4-f30b4d3e3833 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:47.054990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df5aed-735f-4a04-a5c4-f30b4d3e3833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:47.057272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df5aed-735f-4a04-a5c4-f30b4d3e3833 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:47.137259 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7eb9e-bd8f-4afe-b4fc-74829837b5a0 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:47.140154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-7bb7eb9e-bd8f-4afe-b4fc-74829837b5a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:47.144668 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 829/1671] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:12:47 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:12:47.147652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df5aed-735f-4a04-a5c4-f30b4d3e3833 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:47.182385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7eb9e-bd8f-4afe-b4fc-74829837b5a0 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:47.261234 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:47.298761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df5aed-735f-4a04-a5c4-f30b4d3e3833 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:47.309457 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40a14e-50b2-42ae-a2a7-6c5c1493fdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:47.310794 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40a14e-50b2-42ae-a2a7-6c5c1493fdb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:47.312559 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40a14e-50b2-42ae-a2a7-6c5c1493fdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:47.313438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40a14e-50b2-42ae-a2a7-6c5c1493fdb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:47.315103 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40a14e-50b2-42ae-a2a7-6c5c1493fdb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:47.320264 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40a14e-50b2-42ae-a2a7-6c5c1493fdb0 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:47.361134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037b757-c1c4-4f0c-9c46-1ec1c6622f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:47.362704 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037b757-c1c4-4f0c-9c46-1ec1c6622f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:47.363301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037b757-c1c4-4f0c-9c46-1ec1c6622f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:47.363873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037b757-c1c4-4f0c-9c46-1ec1c6622f33 admin admin] PATH_INFO: `/v3/projects/8d844e39eed04b248d2bc6a19d0f41a8` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:47.366674 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037b757-c1c4-4f0c-9c46-1ec1c6622f33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d844e39eed04b248d2bc6a19d0f41a8)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:47.374415 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 830/1672] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:12:47 2022] POST /identity/v3/users => generated 287 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:47.374995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037b757-c1c4-4f0c-9c46-1ec1c6622f33 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:47.377757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40a14e-50b2-42ae-a2a7-6c5c1493fdb0 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:47.411712 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40a14e-50b2-42ae-a2a7-6c5c1493fdb0 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:47.419226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff348af7-aec4-4509-b2cb-dad6afd7fc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:47.419972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff348af7-aec4-4509-b2cb-dad6afd7fc3c admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:47.420447 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff348af7-aec4-4509-b2cb-dad6afd7fc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:47.420899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff348af7-aec4-4509-b2cb-dad6afd7fc3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:47.422068 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff348af7-aec4-4509-b2cb-dad6afd7fc3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:47.424079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff348af7-aec4-4509-b2cb-dad6afd7fc3c admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:47.439516 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 831/1673] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:47.446249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff348af7-aec4-4509-b2cb-dad6afd7fc3c admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:47.521653 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff348af7-aec4-4509-b2cb-dad6afd7fc3c admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:47.548176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-2037b757-c1c4-4f0c-9c46-1ec1c6622f33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d844e39eed04b248d2bc6a19d0f41a8', 'request_id': 'req-2037b757-c1c4-4f0c-9c46-1ec1c6622f33'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:47.570539 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-2037b757-c1c4-4f0c-9c46-1ec1c6622f33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d844e39eed04b248d2bc6a19d0f41a8', 'request_id': 'req-2037b757-c1c4-4f0c-9c46-1ec1c6622f33'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:47.577380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:47.597782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-2037b757-c1c4-4f0c-9c46-1ec1c6622f33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d844e39eed04b248d2bc6a19d0f41a8', 'request_id': 'req-2037b757-c1c4-4f0c-9c46-1ec1c6622f33'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:47.650644 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-2037b757-c1c4-4f0c-9c46-1ec1c6622f33 admin admin] The token cache is being invalidate because project 8d844e39eed04b248d2bc6a19d0f41a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:12:47.651553 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-2037b757-c1c4-4f0c-9c46-1ec1c6622f33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:47.654423 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-2037b757-c1c4-4f0c-9c46-1ec1c6622f33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:47.670338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 843/1674] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:12:47 2022] DELETE /identity/v3/projects/8d844e39eed04b248d2bc6a19d0f41a8 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:47.680836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c02c5f-d4e1-4f4a-86da-5d527d17fa73 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:47.682128 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c02c5f-d4e1-4f4a-86da-5d527d17fa73 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:47.684237 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c02c5f-d4e1-4f4a-86da-5d527d17fa73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:47.706839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f00b91-ac04-4262-84c0-b907b41ada0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:47.708115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f00b91-ac04-4262-84c0-b907b41ada0d admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:47.708765 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f00b91-ac04-4262-84c0-b907b41ada0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:47.709370 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f00b91-ac04-4262-84c0-b907b41ada0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:47.710817 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f00b91-ac04-4262-84c0-b907b41ada0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:47.714718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f00b91-ac04-4262-84c0-b907b41ada0d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:47.739566 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 832/1675] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:47.745431 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f00b91-ac04-4262-84c0-b907b41ada0d admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:47.749145 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-47f00b91-ac04-4262-84c0-b907b41ada0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:47.760760 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-20c02c5f-d4e1-4f4a-86da-5d527d17fa73 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:47.767333 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-20c02c5f-d4e1-4f4a-86da-5d527d17fa73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:47.787863 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f00b91-ac04-4262-84c0-b907b41ada0d admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:47.799550 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d9532-2178-4a76-a34b-0a7bddbe625b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:47.801468 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d9532-2178-4a76-a34b-0a7bddbe625b admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:47.802110 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d9532-2178-4a76-a34b-0a7bddbe625b admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:47.802839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d9532-2178-4a76-a34b-0a7bddbe625b admin admin] PATH_INFO: `/v3/projects/48377ffae6d3402ca336a83bb8024db1/users/915d10d6f93841c291b27056e02d2db8/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:47.816138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 844/1676] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:12:47.817735 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a72f9-b8e1-4c21-b6cf-b5b2e3c10f36 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:47.819665 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a72f9-b8e1-4c21-b6cf-b5b2e3c10f36 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:47.819665 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a72f9-b8e1-4c21-b6cf-b5b2e3c10f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:47.904731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d9532-2178-4a76-a34b-0a7bddbe625b admin admin] RBAC: Authorizing `identity:create_grant(project_id=48377ffae6d3402ca336a83bb8024db1, user_id=915d10d6f93841c291b27056e02d2db8, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:47.911538 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-e95a72f9-b8e1-4c21-b6cf-b5b2e3c10f36 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:47.913645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e95a72f9-b8e1-4c21-b6cf-b5b2e3c10f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:47.921302 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d9532-2178-4a76-a34b-0a7bddbe625b admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:47.977448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 845/1677] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:12:47.994109 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a72f9-b8e1-4c21-b6cf-b5b2e3c10f36 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:48.020001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 833/1678] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:47 2022] PUT /identity/v3/projects/48377ffae6d3402ca336a83bb8024db1/users/915d10d6f93841c291b27056e02d2db8/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:48.040132 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a72f9-b8e1-4c21-b6cf-b5b2e3c10f36 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:48.049767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265ba2b-f7b1-420f-9c6f-80e274a2d2a0 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:48.050433 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265ba2b-f7b1-420f-9c6f-80e274a2d2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:48.051044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265ba2b-f7b1-420f-9c6f-80e274a2d2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:48.114912 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-a265ba2b-f7b1-420f-9c6f-80e274a2d2a0 None None] MFA Rules not processed for user `915d10d6f93841c291b27056e02d2db8`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:48.117759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a265ba2b-f7b1-420f-9c6f-80e274a2d2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:48.297558 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 834/1679] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:48.299540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265ba2b-f7b1-420f-9c6f-80e274a2d2a0 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:48.372717 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd221be-02df-4ccc-a1d7-5c0abac072c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:48.375657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd221be-02df-4ccc-a1d7-5c0abac072c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:48.378482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd221be-02df-4ccc-a1d7-5c0abac072c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:48.380696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd221be-02df-4ccc-a1d7-5c0abac072c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:48.383559 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd221be-02df-4ccc-a1d7-5c0abac072c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:48.388074 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd221be-02df-4ccc-a1d7-5c0abac072c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:48.396516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd221be-02df-4ccc-a1d7-5c0abac072c8 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:48.440777 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265ba2b-f7b1-420f-9c6f-80e274a2d2a0 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:48.446694 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944721a1-45bd-488d-80a9-464aac4d09f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:48.448002 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944721a1-45bd-488d-80a9-464aac4d09f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:48.448002 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944721a1-45bd-488d-80a9-464aac4d09f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:48.448331 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944721a1-45bd-488d-80a9-464aac4d09f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:48.452011 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944721a1-45bd-488d-80a9-464aac4d09f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:48.453085 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944721a1-45bd-488d-80a9-464aac4d09f2 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:48.465475 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 846/1680] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 474 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:48.472347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd221be-02df-4ccc-a1d7-5c0abac072c8 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:48.514724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 835/1681] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:12:48 2022] POST /identity/v3/projects => generated 371 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:48.516002 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944721a1-45bd-488d-80a9-464aac4d09f2 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:48.529237 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:48.544378 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd221be-02df-4ccc-a1d7-5c0abac072c8 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:48.669547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944721a1-45bd-488d-80a9-464aac4d09f2 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:48.688647 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:48.700574 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f79a75-eecf-4c11-8b23-2e1ee18c7454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:48.702444 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f79a75-eecf-4c11-8b23-2e1ee18c7454 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:48.705698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f79a75-eecf-4c11-8b23-2e1ee18c7454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:48.706308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f79a75-eecf-4c11-8b23-2e1ee18c7454 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:48.707585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f79a75-eecf-4c11-8b23-2e1ee18c7454 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:48.710344 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f79a75-eecf-4c11-8b23-2e1ee18c7454 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:48.797996 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d495a42-739a-49eb-b337-ae5c1f171f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:48.799978 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 836/1682] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:12:48 2022] POST /identity/v3/users => generated 292 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:48.801063 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d495a42-739a-49eb-b337-ae5c1f171f26 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:48.802496 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d495a42-739a-49eb-b337-ae5c1f171f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:48.803000 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d495a42-739a-49eb-b337-ae5c1f171f26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:48.805422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d495a42-739a-49eb-b337-ae5c1f171f26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:48.806158 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f79a75-eecf-4c11-8b23-2e1ee18c7454 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:48.810304 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d495a42-739a-49eb-b337-ae5c1f171f26 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:48.853923 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f79a75-eecf-4c11-8b23-2e1ee18c7454 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:48.909449 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886483d4-ddb8-418b-9255-959f0feb5774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:48.910490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886483d4-ddb8-418b-9255-959f0feb5774 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:48.910856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886483d4-ddb8-418b-9255-959f0feb5774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:48.911141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886483d4-ddb8-418b-9255-959f0feb5774 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:48.912340 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886483d4-ddb8-418b-9255-959f0feb5774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:48.914850 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886483d4-ddb8-418b-9255-959f0feb5774 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:48.987675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 847/1683] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:12:48 2022] POST /identity/v3/projects => generated 385 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:48.990142 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d495a42-739a-49eb-b337-ae5c1f171f26 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:48.993322 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0d495a42-739a-49eb-b337-ae5c1f171f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:48.997946 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 837/1684] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:49.000619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886483d4-ddb8-418b-9255-959f0feb5774 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:49.039834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886483d4-ddb8-418b-9255-959f0feb5774 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:49.284167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485384b-52c6-4a0e-8fb5-8645d17b7772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:49.287605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485384b-52c6-4a0e-8fb5-8645d17b7772 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:49.288376 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485384b-52c6-4a0e-8fb5-8645d17b7772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:49.289949 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485384b-52c6-4a0e-8fb5-8645d17b7772 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:49.295763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485384b-52c6-4a0e-8fb5-8645d17b7772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:49.303534 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485384b-52c6-4a0e-8fb5-8645d17b7772 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:49.343264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d495a42-739a-49eb-b337-ae5c1f171f26 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:49.387462 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584ec32-900a-4ded-9614-5bd330364e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:49.388849 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584ec32-900a-4ded-9614-5bd330364e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:49.389364 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584ec32-900a-4ded-9614-5bd330364e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:49.391754 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584ec32-900a-4ded-9614-5bd330364e5b admin admin] PATH_INFO: `/v3/projects/a6ed649585c7475a96ee1eb5ebd60eba` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:49.393123 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584ec32-900a-4ded-9614-5bd330364e5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6ed649585c7475a96ee1eb5ebd60eba)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:49.397712 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584ec32-900a-4ded-9614-5bd330364e5b admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:49.461805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 838/1685] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:12:48 2022] POST /identity/v3/users => generated 293 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:49.462678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485384b-52c6-4a0e-8fb5-8645d17b7772 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:49.516157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485384b-52c6-4a0e-8fb5-8645d17b7772 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:49.536184 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9e5b5-42a8-405c-bcaf-addb60057706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:49.537091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9e5b5-42a8-405c-bcaf-addb60057706 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:49.537401 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9e5b5-42a8-405c-bcaf-addb60057706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:49.537688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9e5b5-42a8-405c-bcaf-addb60057706 admin admin] PATH_INFO: `/v3/projects/4a1093da0f6d4ce69499a561d173fcbd/users/7512357fd3da4e6cb52a490c267fd004/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:49.601987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c9e5b5-42a8-405c-bcaf-addb60057706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a1093da0f6d4ce69499a561d173fcbd, user_id=7512357fd3da4e6cb52a490c267fd004, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:49.611602 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c9e5b5-42a8-405c-bcaf-addb60057706 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:49.671201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f584ec32-900a-4ded-9614-5bd330364e5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6ed649585c7475a96ee1eb5ebd60eba', 'request_id': 'req-f584ec32-900a-4ded-9614-5bd330364e5b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:49.674870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 839/1686] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:49 2022] PUT /identity/v3/projects/4a1093da0f6d4ce69499a561d173fcbd/users/7512357fd3da4e6cb52a490c267fd004/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:49.676498 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9e5b5-42a8-405c-bcaf-addb60057706 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:49.706741 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f584ec32-900a-4ded-9614-5bd330364e5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6ed649585c7475a96ee1eb5ebd60eba', 'request_id': 'req-f584ec32-900a-4ded-9614-5bd330364e5b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:49.734154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f584ec32-900a-4ded-9614-5bd330364e5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6ed649585c7475a96ee1eb5ebd60eba', 'request_id': 'req-f584ec32-900a-4ded-9614-5bd330364e5b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:49.785686 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f584ec32-900a-4ded-9614-5bd330364e5b admin admin] The token cache is being invalidate because project a6ed649585c7475a96ee1eb5ebd60eba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:12:49.786215 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f584ec32-900a-4ded-9614-5bd330364e5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:49.790103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f584ec32-900a-4ded-9614-5bd330364e5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:49.793465 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 848/1687] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:12:48 2022] DELETE /identity/v3/projects/a6ed649585c7475a96ee1eb5ebd60eba => generated 0 bytes in 805 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:49.794821 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584ec32-900a-4ded-9614-5bd330364e5b admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:49.798926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f584ec32-900a-4ded-9614-5bd330364e5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:49.864443 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:49.909739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9e5b5-42a8-405c-bcaf-addb60057706 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:49.923584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21680d7-5c11-406d-be61-ef7d0e7aa58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:49.924583 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21680d7-5c11-406d-be61-ef7d0e7aa58c admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:49.925007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21680d7-5c11-406d-be61-ef7d0e7aa58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:49.925189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21680d7-5c11-406d-be61-ef7d0e7aa58c admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:49.925944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21680d7-5c11-406d-be61-ef7d0e7aa58c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:49.928609 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21680d7-5c11-406d-be61-ef7d0e7aa58c admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:49.957439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 840/1688] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:49.958358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21680d7-5c11-406d-be61-ef7d0e7aa58c admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:49.963122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a21680d7-5c11-406d-be61-ef7d0e7aa58c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:50.011577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:50.035551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21680d7-5c11-406d-be61-ef7d0e7aa58c admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:50.054741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cce172-b911-4394-8000-6e17fb01c23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:50.056978 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cce172-b911-4394-8000-6e17fb01c23d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:50.057543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cce172-b911-4394-8000-6e17fb01c23d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:50.058626 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cce172-b911-4394-8000-6e17fb01c23d admin admin] PATH_INFO: `/v3/projects/2e65cdc3641841ba86f30f6b2bf561a1` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:50.076939 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cce172-b911-4394-8000-6e17fb01c23d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e65cdc3641841ba86f30f6b2bf561a1)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:50.082971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584ec32-900a-4ded-9614-5bd330364e5b admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:50.093715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d19e1-6c3b-4f17-96c1-19a3357cbd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:50.095577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d19e1-6c3b-4f17-96c1-19a3357cbd8c admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:50.096648 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d19e1-6c3b-4f17-96c1-19a3357cbd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:50.097905 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d19e1-6c3b-4f17-96c1-19a3357cbd8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:50.100956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d19e1-6c3b-4f17-96c1-19a3357cbd8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:50.105581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d19e1-6c3b-4f17-96c1-19a3357cbd8c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:50.107198 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cce172-b911-4394-8000-6e17fb01c23d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:50.135230 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 849/1689] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:50.137524 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d19e1-6c3b-4f17-96c1-19a3357cbd8c admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:50.140779 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-726d19e1-6c3b-4f17-96c1-19a3357cbd8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:50.185833 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d19e1-6c3b-4f17-96c1-19a3357cbd8c admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:50.196170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530798f-0294-44d0-87f2-860325356266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:50.199799 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530798f-0294-44d0-87f2-860325356266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:50.200611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530798f-0294-44d0-87f2-860325356266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:50.201330 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530798f-0294-44d0-87f2-860325356266 admin admin] PATH_INFO: `/v3/users/7da9afd0e0284cbd86f75d598d3ca0aa` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:50.204336 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530798f-0294-44d0-87f2-860325356266 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7da9afd0e0284cbd86f75d598d3ca0aa)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:50.210974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530798f-0294-44d0-87f2-860325356266 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:50.319484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c5cce172-b911-4394-8000-6e17fb01c23d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e65cdc3641841ba86f30f6b2bf561a1', 'request_id': 'req-c5cce172-b911-4394-8000-6e17fb01c23d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:50.345963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c5cce172-b911-4394-8000-6e17fb01c23d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e65cdc3641841ba86f30f6b2bf561a1', 'request_id': 'req-c5cce172-b911-4394-8000-6e17fb01c23d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:50.361346 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c5cce172-b911-4394-8000-6e17fb01c23d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e65cdc3641841ba86f30f6b2bf561a1', 'request_id': 'req-c5cce172-b911-4394-8000-6e17fb01c23d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:50.380572 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b530798f-0294-44d0-87f2-860325356266 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7da9afd0e0284cbd86f75d598d3ca0aa'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:50.408392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c5cce172-b911-4394-8000-6e17fb01c23d admin admin] The token cache is being invalidate because project 2e65cdc3641841ba86f30f6b2bf561a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:12:50.408933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c5cce172-b911-4394-8000-6e17fb01c23d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:50.412310 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c5cce172-b911-4394-8000-6e17fb01c23d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:50.418894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 841/1690] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:12:49 2022] DELETE /identity/v3/projects/2e65cdc3641841ba86f30f6b2bf561a1 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:50.427212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b530798f-0294-44d0-87f2-860325356266 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7da9afd0e0284cbd86f75d598d3ca0aa'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:50.433898 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cce172-b911-4394-8000-6e17fb01c23d admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:50.438079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c5cce172-b911-4394-8000-6e17fb01c23d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:50.455561 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b530798f-0294-44d0-87f2-860325356266 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7da9afd0e0284cbd86f75d598d3ca0aa'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:50.459087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b530798f-0294-44d0-87f2-860325356266 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7da9afd0e0284cbd86f75d598d3ca0aa'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:50.463605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b530798f-0294-44d0-87f2-860325356266 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7da9afd0e0284cbd86f75d598d3ca0aa'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:50.560937 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 850/1691] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:12:50 2022] DELETE /identity/v3/users/7da9afd0e0284cbd86f75d598d3ca0aa => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:50.563083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530798f-0294-44d0-87f2-860325356266 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:50.566044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b530798f-0294-44d0-87f2-860325356266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:50.736019 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530798f-0294-44d0-87f2-860325356266 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:50.749995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e8346-d9ec-4ba9-9f5b-bdc20bbd362c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:50.752017 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e8346-d9ec-4ba9-9f5b-bdc20bbd362c admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:50.752842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e8346-d9ec-4ba9-9f5b-bdc20bbd362c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:50.753658 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e8346-d9ec-4ba9-9f5b-bdc20bbd362c admin admin] PATH_INFO: `/v3/projects/4a1093da0f6d4ce69499a561d173fcbd/users/7512357fd3da4e6cb52a490c267fd004/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:50.761630 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e8346-d9ec-4ba9-9f5b-bdc20bbd362c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a1093da0f6d4ce69499a561d173fcbd, user_id=7512357fd3da4e6cb52a490c267fd004, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:50.767878 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e8346-d9ec-4ba9-9f5b-bdc20bbd362c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:50.795290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cce172-b911-4394-8000-6e17fb01c23d admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:50.815177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 851/1692] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:50 2022] PUT /identity/v3/projects/4a1093da0f6d4ce69499a561d173fcbd/users/7512357fd3da4e6cb52a490c267fd004/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:50.820866 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ad6b8-9785-48d2-8796-b299e000bc43 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:50.821906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ad6b8-9785-48d2-8796-b299e000bc43 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:50.822725 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ad6b8-9785-48d2-8796-b299e000bc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:50.956563 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-d41ad6b8-9785-48d2-8796-b299e000bc43 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:50.976159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-d41ad6b8-9785-48d2-8796-b299e000bc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:51.146103 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39690a07-5e59-4a57-b0cc-786b2a4a2c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:51.147179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39690a07-5e59-4a57-b0cc-786b2a4a2c9a admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:51.147679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39690a07-5e59-4a57-b0cc-786b2a4a2c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:51.148156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39690a07-5e59-4a57-b0cc-786b2a4a2c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:51.152939 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39690a07-5e59-4a57-b0cc-786b2a4a2c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:51.163397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39690a07-5e59-4a57-b0cc-786b2a4a2c9a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:51.216283 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:51.227082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 842/1693] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 807 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:51.240724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14ddd6-513b-4d82-a079-f9adc8cfda75 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:51.242582 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14ddd6-513b-4d82-a079-f9adc8cfda75 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:51.244436 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14ddd6-513b-4d82-a079-f9adc8cfda75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:51.284122 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 852/1694] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:51.299036 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ad6b8-9785-48d2-8796-b299e000bc43 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:51.299036 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-d41ad6b8-9785-48d2-8796-b299e000bc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:51.350007 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ad6b8-9785-48d2-8796-b299e000bc43 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:51.367493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:51.376649 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-2a14ddd6-513b-4d82-a079-f9adc8cfda75 None None] MFA Rules not processed for user `7512357fd3da4e6cb52a490c267fd004`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:51.378461 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-2a14ddd6-513b-4d82-a079-f9adc8cfda75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:51.389779 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e79b98-b478-4b9d-b815-63b4468773f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:51.393987 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e79b98-b478-4b9d-b815-63b4468773f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:51.394532 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e79b98-b478-4b9d-b815-63b4468773f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:51.395042 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e79b98-b478-4b9d-b815-63b4468773f5 admin admin] PATH_INFO: `/v3/projects/e11f9598134c4de5b533cabdfc01a185/users/f21f9fb8cf7a43da86b149b018e83765/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:51.440113 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e79b98-b478-4b9d-b815-63b4468773f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e11f9598134c4de5b533cabdfc01a185, user_id=f21f9fb8cf7a43da86b149b018e83765, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:51.449340 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e79b98-b478-4b9d-b815-63b4468773f5 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:51.521298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 853/1695] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:51 2022] PUT /identity/v3/projects/e11f9598134c4de5b533cabdfc01a185/users/f21f9fb8cf7a43da86b149b018e83765/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:51.525985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f4947-fd43-4bc5-832f-aebb87433518 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:51.525985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f4947-fd43-4bc5-832f-aebb87433518 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:51.538855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f4947-fd43-4bc5-832f-aebb87433518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:51.635017 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-5b9f4947-fd43-4bc5-832f-aebb87433518 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:51.639441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5b9f4947-fd43-4bc5-832f-aebb87433518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:51.730545 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 843/1696] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:51 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 503 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:51.733904 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe6594-e2f4-427d-bdc4-0a1bac83033f None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:51.734498 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe6594-e2f4-427d-bdc4-0a1bac83033f None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:51.735475 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe6594-e2f4-427d-bdc4-0a1bac83033f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:51.858857 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-5ffe6594-e2f4-427d-bdc4-0a1bac83033f None None] MFA Rules not processed for user `f21f9fb8cf7a43da86b149b018e83765`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:51.861479 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-5ffe6594-e2f4-427d-bdc4-0a1bac83033f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:51.899053 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 854/1697] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:51.900669 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f4947-fd43-4bc5-832f-aebb87433518 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:51.903195 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5b9f4947-fd43-4bc5-832f-aebb87433518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:52.090861 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 844/1698] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:52.093855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe6594-e2f4-427d-bdc4-0a1bac83033f None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:52.101269 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f4947-fd43-4bc5-832f-aebb87433518 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:52.111118 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e52b1-3353-4429-b10d-7a430d34e34d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:52.113881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e52b1-3353-4429-b10d-7a430d34e34d service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:52.114167 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e52b1-3353-4429-b10d-7a430d34e34d service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:52.114585 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e52b1-3353-4429-b10d-7a430d34e34d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:52.121994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe6594-e2f4-427d-bdc4-0a1bac83033f None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:52.133551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8872ad-08fb-4128-a3c0-3536d99ced69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:52.135530 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8872ad-08fb-4128-a3c0-3536d99ced69 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:52.136926 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8872ad-08fb-4128-a3c0-3536d99ced69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:52.137755 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8872ad-08fb-4128-a3c0-3536d99ced69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:52.139538 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8872ad-08fb-4128-a3c0-3536d99ced69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:52.140364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8872ad-08fb-4128-a3c0-3536d99ced69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:52.144532 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8872ad-08fb-4128-a3c0-3536d99ced69 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:52.181009 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 845/1699] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:52.183339 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8872ad-08fb-4128-a3c0-3536d99ced69 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:52.236289 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8872ad-08fb-4128-a3c0-3536d99ced69 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:52.263571 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab56f364-e5b5-4aac-9200-0147b078b3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:52.265091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56f364-e5b5-4aac-9200-0147b078b3c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:52.265809 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56f364-e5b5-4aac-9200-0147b078b3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:52.267609 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56f364-e5b5-4aac-9200-0147b078b3c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:52.269200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab56f364-e5b5-4aac-9200-0147b078b3c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:52.279032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab56f364-e5b5-4aac-9200-0147b078b3c9 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:52.344633 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e52b1-3353-4429-b10d-7a430d34e34d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:52.347713 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e52b1-3353-4429-b10d-7a430d34e34d service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:52.366744 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 846/1700] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:12:52 2022] POST /identity/v3/projects => generated 385 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:52.366744 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab56f364-e5b5-4aac-9200-0147b078b3c9 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:52.390292 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 855/1701] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:12:51 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:52.391273 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e52b1-3353-4429-b10d-7a430d34e34d service cinder] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:52.398044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b33e52b1-3353-4429-b10d-7a430d34e34d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:52.447685 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e52b1-3353-4429-b10d-7a430d34e34d service cinder] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:52.502038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:52.540653 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:52.558707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab56f364-e5b5-4aac-9200-0147b078b3c9 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:52.571184 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8c160-784b-48fb-b33e-1b1f1da9cecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:52.574307 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8c160-784b-48fb-b33e-1b1f1da9cecb admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:52.574761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8c160-784b-48fb-b33e-1b1f1da9cecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:52.575097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8c160-784b-48fb-b33e-1b1f1da9cecb admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:52.576110 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b8c160-784b-48fb-b33e-1b1f1da9cecb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:52.578549 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b8c160-784b-48fb-b33e-1b1f1da9cecb admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:52.644688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d2d1a-7b9c-4a35-8c66-16493da06020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:52.645604 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d2d1a-7b9c-4a35-8c66-16493da06020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:52.646049 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d2d1a-7b9c-4a35-8c66-16493da06020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:52.646456 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d2d1a-7b9c-4a35-8c66-16493da06020 admin admin] PATH_INFO: `/v3/projects/3759586c5a8c4fe4a7c2310513dac4c3` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:52.656115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d2d1a-7b9c-4a35-8c66-16493da06020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3759586c5a8c4fe4a7c2310513dac4c3)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:52.660085 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d2d1a-7b9c-4a35-8c66-16493da06020 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:52.713605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 847/1702] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:12:52 2022] POST /identity/v3/projects => generated 387 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:52.713605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8c160-784b-48fb-b33e-1b1f1da9cecb admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:52.874050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-c32d2d1a-7b9c-4a35-8c66-16493da06020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3759586c5a8c4fe4a7c2310513dac4c3', 'request_id': 'req-c32d2d1a-7b9c-4a35-8c66-16493da06020'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:52.901616 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-c32d2d1a-7b9c-4a35-8c66-16493da06020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3759586c5a8c4fe4a7c2310513dac4c3', 'request_id': 'req-c32d2d1a-7b9c-4a35-8c66-16493da06020'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:52.916580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-c32d2d1a-7b9c-4a35-8c66-16493da06020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3759586c5a8c4fe4a7c2310513dac4c3', 'request_id': 'req-c32d2d1a-7b9c-4a35-8c66-16493da06020'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:52.942331 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-c32d2d1a-7b9c-4a35-8c66-16493da06020 admin admin] The token cache is being invalidate because project 3759586c5a8c4fe4a7c2310513dac4c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:12:52.942993 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-c32d2d1a-7b9c-4a35-8c66-16493da06020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:52.944303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-c32d2d1a-7b9c-4a35-8c66-16493da06020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:52.968054 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 856/1703] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:12:52 2022] DELETE /identity/v3/projects/3759586c5a8c4fe4a7c2310513dac4c3 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:52.969747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d2d1a-7b9c-4a35-8c66-16493da06020 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:52.972407 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-c32d2d1a-7b9c-4a35-8c66-16493da06020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:53.005437 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8c160-784b-48fb-b33e-1b1f1da9cecb admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:53.190125 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d2d1a-7b9c-4a35-8c66-16493da06020 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:53.199298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ccea5-bc08-4593-9923-3fea6dfaa6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:53.200149 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ccea5-bc08-4593-9923-3fea6dfaa6c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:53.201152 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ccea5-bc08-4593-9923-3fea6dfaa6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:53.201452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ccea5-bc08-4593-9923-3fea6dfaa6c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:53.202570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ccea5-bc08-4593-9923-3fea6dfaa6c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:53.206114 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ccea5-bc08-4593-9923-3fea6dfaa6c0 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:53.288202 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f51b1-f028-476b-a2a3-f9c292cc7a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:53.289703 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f51b1-f028-476b-a2a3-f9c292cc7a59 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:53.290569 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f51b1-f028-476b-a2a3-f9c292cc7a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:53.291123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f51b1-f028-476b-a2a3-f9c292cc7a59 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:53.298327 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 857/1704] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:12:52 2022] POST /identity/v3/users => generated 294 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:53.299125 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f51b1-f028-476b-a2a3-f9c292cc7a59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:53.304103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ccea5-bc08-4593-9923-3fea6dfaa6c0 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:53.311644 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f51b1-f028-476b-a2a3-f9c292cc7a59 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:53.315168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5d4ccea5-bc08-4593-9923-3fea6dfaa6c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:53.367369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 848/1705] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:12:52 2022] POST /identity/v3/users => generated 299 bytes in 656 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:53.370043 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538130ff-a02c-4073-9693-3cf372c5d869 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:53.370463 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538130ff-a02c-4073-9693-3cf372c5d869 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:53.370718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538130ff-a02c-4073-9693-3cf372c5d869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:53.425962 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-538130ff-a02c-4073-9693-3cf372c5d869 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:53.430615 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-538130ff-a02c-4073-9693-3cf372c5d869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:53.457853 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 849/1706] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:53.468646 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538130ff-a02c-4073-9693-3cf372c5d869 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:53.505775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538130ff-a02c-4073-9693-3cf372c5d869 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:53.517198 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:53.532273 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d5a23-8774-4f92-b2b0-5bf1d04872a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:53.533087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d5a23-8774-4f92-b2b0-5bf1d04872a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:53.534554 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d5a23-8774-4f92-b2b0-5bf1d04872a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:53.534891 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d5a23-8774-4f92-b2b0-5bf1d04872a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:53.535748 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d5a23-8774-4f92-b2b0-5bf1d04872a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:53.537947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d5a23-8774-4f92-b2b0-5bf1d04872a2 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:53.576726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ccea5-bc08-4593-9923-3fea6dfaa6c0 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:53.578665 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 850/1707] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:53.579819 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d5a23-8774-4f92-b2b0-5bf1d04872a2 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:53.583460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-6b2d5a23-8774-4f92-b2b0-5bf1d04872a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:53.588321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde22f9-9192-474f-986d-2ab7409d3c2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:53.590970 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde22f9-9192-474f-986d-2ab7409d3c2c service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:53.591322 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde22f9-9192-474f-986d-2ab7409d3c2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:53.591589 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde22f9-9192-474f-986d-2ab7409d3c2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:53.595876 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-fbde22f9-9192-474f-986d-2ab7409d3c2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:53.613777 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d5a23-8774-4f92-b2b0-5bf1d04872a2 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:53.635772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203cf116-30ad-40a4-92f9-31419e23ea26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:53.637296 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cf116-30ad-40a4-92f9-31419e23ea26 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:53.637580 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cf116-30ad-40a4-92f9-31419e23ea26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:53.637850 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cf116-30ad-40a4-92f9-31419e23ea26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:53.638640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203cf116-30ad-40a4-92f9-31419e23ea26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:53.640844 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203cf116-30ad-40a4-92f9-31419e23ea26 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:53.676815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 851/1708] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:53.679518 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fc824-2b08-4535-acb8-86f9d23c4e8e None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:53.679895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fc824-2b08-4535-acb8-86f9d23c4e8e None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:53.680185 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fc824-2b08-4535-acb8-86f9d23c4e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:53.719844 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-711fc824-2b08-4535-acb8-86f9d23c4e8e None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:53.722624 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-711fc824-2b08-4535-acb8-86f9d23c4e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:53.740929 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 852/1709] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:53.746317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711fc824-2b08-4535-acb8-86f9d23c4e8e None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:53.772071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711fc824-2b08-4535-acb8-86f9d23c4e8e None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:53.804171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c8f14-9423-4089-b6d2-2ce75f053f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:53.805540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c8f14-9423-4089-b6d2-2ce75f053f99 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:53.806084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c8f14-9423-4089-b6d2-2ce75f053f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:53.806534 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c8f14-9423-4089-b6d2-2ce75f053f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:53.812944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c8f14-9423-4089-b6d2-2ce75f053f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:53.821763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c8f14-9423-4089-b6d2-2ce75f053f99 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:53.873333 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde22f9-9192-474f-986d-2ab7409d3c2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:53.877948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde22f9-9192-474f-986d-2ab7409d3c2c service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:53.880884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 853/1710] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:53.882494 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c8f14-9423-4089-b6d2-2ce75f053f99 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:53.930970 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 858/1711] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:12:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:53.932604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c8f14-9423-4089-b6d2-2ce75f053f99 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:53.935024 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde22f9-9192-474f-986d-2ab7409d3c2c service cinder] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:53.955310 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938166f5-d1eb-4b45-a8f2-ddb138f3cc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:53.956570 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938166f5-d1eb-4b45-a8f2-ddb138f3cc0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:53.961084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938166f5-d1eb-4b45-a8f2-ddb138f3cc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:53.964719 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938166f5-d1eb-4b45-a8f2-ddb138f3cc0a admin admin] PATH_INFO: `/v3/projects/ef208ca2fedf4f91812d06e8e90fb29f/users/aa6781c391984c8a9f76de2b43739b8a/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:53.985624 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde22f9-9192-474f-986d-2ab7409d3c2c service cinder] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:54.006468 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a55ddc-f8fe-489b-8646-b38d274c88e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:54.007590 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a55ddc-f8fe-489b-8646-b38d274c88e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:54.008055 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a55ddc-f8fe-489b-8646-b38d274c88e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:54.014053 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a55ddc-f8fe-489b-8646-b38d274c88e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:54.014053 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a55ddc-f8fe-489b-8646-b38d274c88e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:54.014440 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a55ddc-f8fe-489b-8646-b38d274c88e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:54.044731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a55ddc-f8fe-489b-8646-b38d274c88e1 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:54.053563 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938166f5-d1eb-4b45-a8f2-ddb138f3cc0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef208ca2fedf4f91812d06e8e90fb29f, user_id=aa6781c391984c8a9f76de2b43739b8a, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:54.065829 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938166f5-d1eb-4b45-a8f2-ddb138f3cc0a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:54.116814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 854/1712] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:53 2022] PUT /identity/v3/projects/ef208ca2fedf4f91812d06e8e90fb29f/users/aa6781c391984c8a9f76de2b43739b8a/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:54.119169 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938166f5-d1eb-4b45-a8f2-ddb138f3cc0a admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:54.133032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:54.135628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 859/1713] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:54.140207 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a55ddc-f8fe-489b-8646-b38d274c88e1 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:54.212243 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a55ddc-f8fe-489b-8646-b38d274c88e1 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:54.396999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938166f5-d1eb-4b45-a8f2-ddb138f3cc0a admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:54.428635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6e499-9729-4208-82e9-3bfb23d59920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:54.430196 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6e499-9729-4208-82e9-3bfb23d59920 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:54.430982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6e499-9729-4208-82e9-3bfb23d59920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:54.432952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6e499-9729-4208-82e9-3bfb23d59920 admin admin] PATH_INFO: `/v3/projects/624eecda61944ff1a74a880ace535a77/users/2aa9621d66a94553b363428cf3130249/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:54.539638 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6e499-9729-4208-82e9-3bfb23d59920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=624eecda61944ff1a74a880ace535a77, user_id=2aa9621d66a94553b363428cf3130249, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:54.552033 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6e499-9729-4208-82e9-3bfb23d59920 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:54.572035 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c72d0-6652-4abf-a655-e71625c6bd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:54.573212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c72d0-6652-4abf-a655-e71625c6bd40 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:54.573764 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c72d0-6652-4abf-a655-e71625c6bd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:54.574242 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c72d0-6652-4abf-a655-e71625c6bd40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:54.575590 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c72d0-6652-4abf-a655-e71625c6bd40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:54.578930 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c72d0-6652-4abf-a655-e71625c6bd40 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:54.589148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 855/1714] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:54 2022] PUT /identity/v3/projects/624eecda61944ff1a74a880ace535a77/users/2aa9621d66a94553b363428cf3130249/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:54.593721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6e499-9729-4208-82e9-3bfb23d59920 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:54.628923 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 860/1715] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 492 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:54.642964 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4330546-643c-4709-b743-358cfbd3d702 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:54.643482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4330546-643c-4709-b743-358cfbd3d702 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:54.643969 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4330546-643c-4709-b743-358cfbd3d702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:54.764766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-b4330546-643c-4709-b743-358cfbd3d702 None None] MFA Rules not processed for user `2aa9621d66a94553b363428cf3130249`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:54.767065 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b4330546-643c-4709-b743-358cfbd3d702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:54.896125 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6e499-9729-4208-82e9-3bfb23d59920 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:54.921679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b268d-8d97-4c38-aa74-b5ec770733a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:54.937483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b268d-8d97-4c38-aa74-b5ec770733a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:54.938145 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b268d-8d97-4c38-aa74-b5ec770733a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:54.938638 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b268d-8d97-4c38-aa74-b5ec770733a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:54.939976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b268d-8d97-4c38-aa74-b5ec770733a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:54.959278 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b268d-8d97-4c38-aa74-b5ec770733a4 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:55.062141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 856/1716] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:12:54 2022] POST /identity/v3/projects => generated 391 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:55.067127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b268d-8d97-4c38-aa74-b5ec770733a4 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:55.134474 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 861/1717] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:55.135608 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4330546-643c-4709-b743-358cfbd3d702 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:55.149352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:55.179068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4330546-643c-4709-b743-358cfbd3d702 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:55.371115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5aa021-84e0-4c59-a0bf-62eae85f0470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:55.371702 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5aa021-84e0-4c59-a0bf-62eae85f0470 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:55.371992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5aa021-84e0-4c59-a0bf-62eae85f0470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:55.372587 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5aa021-84e0-4c59-a0bf-62eae85f0470 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:55.373140 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5aa021-84e0-4c59-a0bf-62eae85f0470 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:55.375330 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5aa021-84e0-4c59-a0bf-62eae85f0470 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:55.389927 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b268d-8d97-4c38-aa74-b5ec770733a4 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:55.408617 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9c593-d6c9-444e-bd52-e563a29a9558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:55.412101 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9c593-d6c9-444e-bd52-e563a29a9558 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:55.420366 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9c593-d6c9-444e-bd52-e563a29a9558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:55.422772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9c593-d6c9-444e-bd52-e563a29a9558 admin admin] PATH_INFO: `/v3/projects/ef208ca2fedf4f91812d06e8e90fb29f/users/aa6781c391984c8a9f76de2b43739b8a/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:55.431975 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9c593-d6c9-444e-bd52-e563a29a9558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef208ca2fedf4f91812d06e8e90fb29f, user_id=aa6781c391984c8a9f76de2b43739b8a, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:55.441013 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9c593-d6c9-444e-bd52-e563a29a9558 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:55.447251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 862/1718] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:12:55 2022] POST /identity/v3/users => generated 296 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:55.448159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5aa021-84e0-4c59-a0bf-62eae85f0470 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:55.467939 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 857/1719] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:55 2022] PUT /identity/v3/projects/ef208ca2fedf4f91812d06e8e90fb29f/users/aa6781c391984c8a9f76de2b43739b8a/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:55.470724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9c593-d6c9-444e-bd52-e563a29a9558 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:55.486121 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5aa021-84e0-4c59-a0bf-62eae85f0470 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:55.508643 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:55.634268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e944839-cf04-4e78-92b2-099ba4dd0a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:55.635653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e944839-cf04-4e78-92b2-099ba4dd0a93 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:55.636282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e944839-cf04-4e78-92b2-099ba4dd0a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:55.636746 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e944839-cf04-4e78-92b2-099ba4dd0a93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:55.638109 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e944839-cf04-4e78-92b2-099ba4dd0a93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:55.653183 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e944839-cf04-4e78-92b2-099ba4dd0a93 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:55.750167 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 863/1720] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:12:55 2022] POST /identity/v3/projects => generated 387 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:55.753311 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2f7ab-e3e5-4565-8b76-4a1bf70f5e67 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:55.754139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2f7ab-e3e5-4565-8b76-4a1bf70f5e67 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:55.754817 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2f7ab-e3e5-4565-8b76-4a1bf70f5e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:55.781752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9c593-d6c9-444e-bd52-e563a29a9558 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:55.817356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-ccd2f7ab-e3e5-4565-8b76-4a1bf70f5e67 None None] MFA Rules not processed for user `aa6781c391984c8a9f76de2b43739b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:55.820125 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ccd2f7ab-e3e5-4565-8b76-4a1bf70f5e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:55.942375 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e7cfc-8df4-433b-8823-a3413aa1bb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:55.943127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e7cfc-8df4-433b-8823-a3413aa1bb6d admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:55.943427 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e7cfc-8df4-433b-8823-a3413aa1bb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:55.943703 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e7cfc-8df4-433b-8823-a3413aa1bb6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:55.944490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e7cfc-8df4-433b-8823-a3413aa1bb6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:55.951283 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e7cfc-8df4-433b-8823-a3413aa1bb6d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:55.992231 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 858/1721] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 523 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:55.994487 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e7cfc-8df4-433b-8823-a3413aa1bb6d admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:56.017425 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 864/1722] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:56.019126 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2f7ab-e3e5-4565-8b76-4a1bf70f5e67 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:56.030674 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:56.043750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2f7ab-e3e5-4565-8b76-4a1bf70f5e67 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:56.045410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e7cfc-8df4-433b-8823-a3413aa1bb6d admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:56.069134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec523f-ea53-4bee-bc8f-85506e11dc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:56.071409 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec523f-ea53-4bee-bc8f-85506e11dc65 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:56.072111 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec523f-ea53-4bee-bc8f-85506e11dc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:56.073126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec523f-ea53-4bee-bc8f-85506e11dc65 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:56.076364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec523f-ea53-4bee-bc8f-85506e11dc65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:56.083420 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c0625-d461-4637-9ac1-fa8e46690b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:56.084979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c0625-d461-4637-9ac1-fa8e46690b78 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:56.085494 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c0625-d461-4637-9ac1-fa8e46690b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:56.085976 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c0625-d461-4637-9ac1-fa8e46690b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:56.087462 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec523f-ea53-4bee-bc8f-85506e11dc65 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:56.091568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c0625-d461-4637-9ac1-fa8e46690b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:56.105704 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c0625-d461-4637-9ac1-fa8e46690b78 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:56.144707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 865/1723] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:56.146482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c0625-d461-4637-9ac1-fa8e46690b78 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:56.166777 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 859/1724] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:12:55 2022] POST /identity/v3/users => generated 300 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:56.173523 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec523f-ea53-4bee-bc8f-85506e11dc65 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:56.205946 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec523f-ea53-4bee-bc8f-85506e11dc65 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:56.220593 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c7d5c-a84c-4513-a457-0a9a58da6f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:56.221584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c7d5c-a84c-4513-a457-0a9a58da6f4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:56.222094 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c7d5c-a84c-4513-a457-0a9a58da6f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:56.222518 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c7d5c-a84c-4513-a457-0a9a58da6f4f admin admin] PATH_INFO: `/v3/projects/cf289e1122ba465082fd2f9b7f5d388a/users/d802c9d3c96d4009aefbfdde9c365543/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:56.291289 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c0625-d461-4637-9ac1-fa8e46690b78 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:56.293864 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c7d5c-a84c-4513-a457-0a9a58da6f4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf289e1122ba465082fd2f9b7f5d388a, user_id=d802c9d3c96d4009aefbfdde9c365543, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:56.303535 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c7d5c-a84c-4513-a457-0a9a58da6f4f admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:56.310213 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd4a4c-337a-4706-858b-51ce76c0eaa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:56.313536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd4a4c-337a-4706-858b-51ce76c0eaa2 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:56.316489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd4a4c-337a-4706-858b-51ce76c0eaa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:56.317075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd4a4c-337a-4706-858b-51ce76c0eaa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:56.352808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 860/1725] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:56 2022] PUT /identity/v3/projects/cf289e1122ba465082fd2f9b7f5d388a/users/d802c9d3c96d4009aefbfdde9c365543/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:56.353947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c7d5c-a84c-4513-a457-0a9a58da6f4f admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:56.388306 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd4a4c-337a-4706-858b-51ce76c0eaa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:56.391257 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd4a4c-337a-4706-858b-51ce76c0eaa2 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:56.437610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 866/1726] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:12:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:56.448420 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88300f-34af-4700-bdf5-4c213e44f120 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:56.451361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88300f-34af-4700-bdf5-4c213e44f120 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:56.451835 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88300f-34af-4700-bdf5-4c213e44f120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:56.614972 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-7c88300f-34af-4700-bdf5-4c213e44f120 None None] MFA Rules not processed for user `d802c9d3c96d4009aefbfdde9c365543`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:56.622606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-7c88300f-34af-4700-bdf5-4c213e44f120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:56.782651 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c7d5c-a84c-4513-a457-0a9a58da6f4f admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:56.826080 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1956d5-b828-48fa-af41-21deab8b0de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:56.827458 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1956d5-b828-48fa-af41-21deab8b0de8 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:56.828130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1956d5-b828-48fa-af41-21deab8b0de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:56.828853 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1956d5-b828-48fa-af41-21deab8b0de8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:56.831356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1956d5-b828-48fa-af41-21deab8b0de8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:56.836488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1956d5-b828-48fa-af41-21deab8b0de8 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:56.869959 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 867/1727] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 431 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:56.872632 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88300f-34af-4700-bdf5-4c213e44f120 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:56.897359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88300f-34af-4700-bdf5-4c213e44f120 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:56.910414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046c15b-dded-4587-bf7d-a6e9d50c5a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:56.912357 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046c15b-dded-4587-bf7d-a6e9d50c5a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:56.919882 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046c15b-dded-4587-bf7d-a6e9d50c5a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:56.921294 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046c15b-dded-4587-bf7d-a6e9d50c5a85 admin admin] PATH_INFO: `/v3/users/f21f9fb8cf7a43da86b149b018e83765` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:56.924387 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046c15b-dded-4587-bf7d-a6e9d50c5a85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f21f9fb8cf7a43da86b149b018e83765)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:56.925956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 861/1728] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 572 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:56.929900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046c15b-dded-4587-bf7d-a6e9d50c5a85 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:56.930675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1956d5-b828-48fa-af41-21deab8b0de8 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:56.969950 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:57.031895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1956d5-b828-48fa-af41-21deab8b0de8 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:57.052277 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9dfd71-35eb-42df-b6bf-7ad9475974ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:57.053183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dfd71-35eb-42df-b6bf-7ad9475974ff admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:57.053756 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dfd71-35eb-42df-b6bf-7ad9475974ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:57.054139 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dfd71-35eb-42df-b6bf-7ad9475974ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:57.055037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9dfd71-35eb-42df-b6bf-7ad9475974ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:57.060604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9dfd71-35eb-42df-b6bf-7ad9475974ff admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:57.111968 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-3046c15b-dded-4587-bf7d-a6e9d50c5a85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f21f9fb8cf7a43da86b149b018e83765'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:57.139541 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-3046c15b-dded-4587-bf7d-a6e9d50c5a85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f21f9fb8cf7a43da86b149b018e83765'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:57.163197 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-3046c15b-dded-4587-bf7d-a6e9d50c5a85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f21f9fb8cf7a43da86b149b018e83765'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:57.163984 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-3046c15b-dded-4587-bf7d-a6e9d50c5a85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f21f9fb8cf7a43da86b149b018e83765'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:57.164629 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-3046c15b-dded-4587-bf7d-a6e9d50c5a85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f21f9fb8cf7a43da86b149b018e83765'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:12:57.173186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 862/1729] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:12:56 2022] POST /identity/v3/projects => generated 391 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:57.175194 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9dfd71-35eb-42df-b6bf-7ad9475974ff admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:57.179592 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ef9dfd71-35eb-42df-b6bf-7ad9475974ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:57.213189 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:57.229012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 868/1730] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:12:56 2022] DELETE /identity/v3/users/f21f9fb8cf7a43da86b149b018e83765 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:57.229012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046c15b-dded-4587-bf7d-a6e9d50c5a85 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:57.233333 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3046c15b-dded-4587-bf7d-a6e9d50c5a85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:57.476037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9dfd71-35eb-42df-b6bf-7ad9475974ff admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:57.609113 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046c15b-dded-4587-bf7d-a6e9d50c5a85 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:57.643854 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b81e4e-23d6-4fdf-bc3d-1a1186d5e57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:57.663172 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b81e4e-23d6-4fdf-bc3d-1a1186d5e57f admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:57.665220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b81e4e-23d6-4fdf-bc3d-1a1186d5e57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:57.665750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b81e4e-23d6-4fdf-bc3d-1a1186d5e57f admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:57.670850 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b81e4e-23d6-4fdf-bc3d-1a1186d5e57f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:57.676130 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b81e4e-23d6-4fdf-bc3d-1a1186d5e57f admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:57.786625 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 869/1731] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:12:57 2022] POST /identity/v3/users => generated 302 bytes in 561 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:57.798903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6257e-ca0f-4c51-9a32-4c30a7b40a29 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:57.805056 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6257e-ca0f-4c51-9a32-4c30a7b40a29 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:57.809326 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6257e-ca0f-4c51-9a32-4c30a7b40a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:57.818959 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829448d2-dbf2-4585-b44e-85df060341fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:57.823392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829448d2-dbf2-4585-b44e-85df060341fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:57.823987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829448d2-dbf2-4585-b44e-85df060341fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:57.824532 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829448d2-dbf2-4585-b44e-85df060341fc admin admin] PATH_INFO: `/v3/projects/8b6014b2454d4e09ac4b5aa01a751222/users/46bbc8aecaab4abebf0db6aae42a834b/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:57.927385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-5ea6257e-ca0f-4c51-9a32-4c30a7b40a29 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:57.938843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5ea6257e-ca0f-4c51-9a32-4c30a7b40a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:57.984125 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829448d2-dbf2-4585-b44e-85df060341fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b6014b2454d4e09ac4b5aa01a751222, user_id=46bbc8aecaab4abebf0db6aae42a834b, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:58.003943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 870/1732] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:58.013386 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6257e-ca0f-4c51-9a32-4c30a7b40a29 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:58.032822 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829448d2-dbf2-4585-b44e-85df060341fc admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:58.067140 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6257e-ca0f-4c51-9a32-4c30a7b40a29 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:58.093524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 863/1733] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:57 2022] PUT /identity/v3/projects/8b6014b2454d4e09ac4b5aa01a751222/users/46bbc8aecaab4abebf0db6aae42a834b/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 912 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:58.096551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829448d2-dbf2-4585-b44e-85df060341fc admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:58.100789 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-829448d2-dbf2-4585-b44e-85df060341fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:58.272274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829448d2-dbf2-4585-b44e-85df060341fc admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:58.281452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbcc12e-d4f3-4e2e-8a36-c57a46f78aec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:58.282651 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcc12e-d4f3-4e2e-8a36-c57a46f78aec service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:58.283150 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcc12e-d4f3-4e2e-8a36-c57a46f78aec service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:58.284410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcc12e-d4f3-4e2e-8a36-c57a46f78aec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:58.357782 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:58.426211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4b0ee-55a9-4f1f-a434-2b179808ae3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:58.428385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4b0ee-55a9-4f1f-a434-2b179808ae3a admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:58.429151 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4b0ee-55a9-4f1f-a434-2b179808ae3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:58.429728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4b0ee-55a9-4f1f-a434-2b179808ae3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:58.431386 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4b0ee-55a9-4f1f-a434-2b179808ae3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:58.436314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4b0ee-55a9-4f1f-a434-2b179808ae3a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:58.438769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbcc12e-d4f3-4e2e-8a36-c57a46f78aec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:58.442152 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbcc12e-d4f3-4e2e-8a36-c57a46f78aec service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:58.466240 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 864/1734] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:12:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:58.468361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbcc12e-d4f3-4e2e-8a36-c57a46f78aec service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:58.483310 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 871/1735] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 479 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:58.492542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbcc12e-d4f3-4e2e-8a36-c57a46f78aec service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:58.499798 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4b0ee-55a9-4f1f-a434-2b179808ae3a admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:58.527493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb2113-2f98-4ec1-b88d-de5d4891bbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:58.527493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb2113-2f98-4ec1-b88d-de5d4891bbb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:58.527493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb2113-2f98-4ec1-b88d-de5d4891bbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:58.527493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb2113-2f98-4ec1-b88d-de5d4891bbb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:58.529666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb2113-2f98-4ec1-b88d-de5d4891bbb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:58.533684 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb2113-2f98-4ec1-b88d-de5d4891bbb2 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:58.567899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4b0ee-55a9-4f1f-a434-2b179808ae3a admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:58.582115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 865/1736] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:58.584169 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb2113-2f98-4ec1-b88d-de5d4891bbb2 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:58.595502 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58979a01-2223-4760-be0e-d025d505bddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:58.596700 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58979a01-2223-4760-be0e-d025d505bddf admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:58.597140 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58979a01-2223-4760-be0e-d025d505bddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:58.597574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58979a01-2223-4760-be0e-d025d505bddf admin admin] PATH_INFO: `/v3/projects/edc96fdfcc5b46209755425c08260a52/users/adb2d3a805d041c293585bf50f6f0ca3/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:58.606957 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb2113-2f98-4ec1-b88d-de5d4891bbb2 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:58.640810 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe56ae72-88bd-4a9c-94a2-f2fcd2813fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:58.642439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56ae72-88bd-4a9c-94a2-f2fcd2813fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:58.643053 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56ae72-88bd-4a9c-94a2-f2fcd2813fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:58.643649 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56ae72-88bd-4a9c-94a2-f2fcd2813fc2 admin admin] PATH_INFO: `/v3/projects/8b6014b2454d4e09ac4b5aa01a751222/users/46bbc8aecaab4abebf0db6aae42a834b/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:58.653865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe56ae72-88bd-4a9c-94a2-f2fcd2813fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b6014b2454d4e09ac4b5aa01a751222, user_id=46bbc8aecaab4abebf0db6aae42a834b, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:58.659536 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe56ae72-88bd-4a9c-94a2-f2fcd2813fc2 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:58.699909 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58979a01-2223-4760-be0e-d025d505bddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=edc96fdfcc5b46209755425c08260a52, user_id=adb2d3a805d041c293585bf50f6f0ca3, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:58.720549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58979a01-2223-4760-be0e-d025d505bddf admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:58.726386 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 866/1737] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:58 2022] PUT /identity/v3/projects/8b6014b2454d4e09ac4b5aa01a751222/users/46bbc8aecaab4abebf0db6aae42a834b/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:58.736625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a9c68-59c4-4ae8-9ace-f1196c734b8c None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:58.737202 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a9c68-59c4-4ae8-9ace-f1196c734b8c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:58.737773 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a9c68-59c4-4ae8-9ace-f1196c734b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:58.793891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 872/1738] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:58 2022] PUT /identity/v3/projects/edc96fdfcc5b46209755425c08260a52/users/adb2d3a805d041c293585bf50f6f0ca3/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:12:58.797989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58979a01-2223-4760-be0e-d025d505bddf admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:58.827587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-622a9c68-59c4-4ae8-9ace-f1196c734b8c None None] MFA Rules not processed for user `46bbc8aecaab4abebf0db6aae42a834b`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:58.830358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-622a9c68-59c4-4ae8-9ace-f1196c734b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:59.076964 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:59.172698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 867/1739] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:12:59.180139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58979a01-2223-4760-be0e-d025d505bddf admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:59.191931 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223b297-df5a-4f59-aa12-70ef64aedcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:59.193081 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223b297-df5a-4f59-aa12-70ef64aedcef admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:59.193695 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223b297-df5a-4f59-aa12-70ef64aedcef admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:59.194418 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223b297-df5a-4f59-aa12-70ef64aedcef admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:59.196108 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223b297-df5a-4f59-aa12-70ef64aedcef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:59.200337 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223b297-df5a-4f59-aa12-70ef64aedcef admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:59.233493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a9c68-59c4-4ae8-9ace-f1196c734b8c None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:59.243542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-622a9c68-59c4-4ae8-9ace-f1196c734b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:59.250983 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 873/1740] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 456 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:12:59.253548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223b297-df5a-4f59-aa12-70ef64aedcef admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:59.334202 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223b297-df5a-4f59-aa12-70ef64aedcef admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:59.353596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cff97-de89-498f-8647-cc941bb659d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:59.355398 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cff97-de89-498f-8647-cc941bb659d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:59.356727 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cff97-de89-498f-8647-cc941bb659d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:59.357271 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cff97-de89-498f-8647-cc941bb659d4 admin admin] PATH_INFO: `/v3/projects/edc96fdfcc5b46209755425c08260a52/users/adb2d3a805d041c293585bf50f6f0ca3/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:59.362814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190cff97-de89-498f-8647-cc941bb659d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edc96fdfcc5b46209755425c08260a52, user_id=adb2d3a805d041c293585bf50f6f0ca3, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:59.369629 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190cff97-de89-498f-8647-cc941bb659d4 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:59.414156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 874/1741] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:12:59 2022] PUT /identity/v3/projects/edc96fdfcc5b46209755425c08260a52/users/adb2d3a805d041c293585bf50f6f0ca3/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:59.444092 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56c0c0-a723-4f11-851d-1ce5e34753e1 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:59.445217 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56c0c0-a723-4f11-851d-1ce5e34753e1 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:59.445664 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56c0c0-a723-4f11-851d-1ce5e34753e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:59.549016 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a9c68-59c4-4ae8-9ace-f1196c734b8c None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:59.558428 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-1b56c0c0-a723-4f11-851d-1ce5e34753e1 None None] MFA Rules not processed for user `adb2d3a805d041c293585bf50f6f0ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:59.560687 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-1b56c0c0-a723-4f11-851d-1ce5e34753e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:59.790713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:59.878805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb1925-e19c-42b0-a6b4-3ba9e3f33d0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:59.882533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb1925-e19c-42b0-a6b4-3ba9e3f33d0f service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:59.883139 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb1925-e19c-42b0-a6b4-3ba9e3f33d0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:59.883630 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb1925-e19c-42b0-a6b4-3ba9e3f33d0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:59.970393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 875/1742] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:12:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 526 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:00.085648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb1925-e19c-42b0-a6b4-3ba9e3f33d0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:00.099837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb1925-e19c-42b0-a6b4-3ba9e3f33d0f service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:00.140268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 868/1743] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:12:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 908 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:00.309328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56c0c0-a723-4f11-851d-1ce5e34753e1 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:00.311899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-1b56c0c0-a723-4f11-851d-1ce5e34753e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:00.359989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a03f9b-34d6-4587-8d6c-0bf0c5924bfe None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:00.360859 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a03f9b-34d6-4587-8d6c-0bf0c5924bfe None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:00.361622 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a03f9b-34d6-4587-8d6c-0bf0c5924bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:00.423512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56c0c0-a723-4f11-851d-1ce5e34753e1 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:00.457921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9127ba-98be-4a8c-8268-fd9f97cc48d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:00.459981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9127ba-98be-4a8c-8268-fd9f97cc48d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:00.463911 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9127ba-98be-4a8c-8268-fd9f97cc48d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:00.466193 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9127ba-98be-4a8c-8268-fd9f97cc48d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:00.533987 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9127ba-98be-4a8c-8268-fd9f97cc48d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:00.542283 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-e3a03f9b-34d6-4587-8d6c-0bf0c5924bfe None None] MFA Rules not processed for user `3c18437883ce42369c7563224c208523`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:00.545255 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e3a03f9b-34d6-4587-8d6c-0bf0c5924bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:00.573099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9127ba-98be-4a8c-8268-fd9f97cc48d7 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:00.676850 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 876/1744] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:00.933186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 869/1745] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 575 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:13:00.965041 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9127ba-98be-4a8c-8268-fd9f97cc48d7 service cinder] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:00.967419 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4f9127ba-98be-4a8c-8268-fd9f97cc48d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:01.076837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a03f9b-34d6-4587-8d6c-0bf0c5924bfe None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:01.136766 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a03f9b-34d6-4587-8d6c-0bf0c5924bfe None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:01.163812 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ba806-99ff-491b-af8d-39c7baa566e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:01.165465 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ba806-99ff-491b-af8d-39c7baa566e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:01.166270 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ba806-99ff-491b-af8d-39c7baa566e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:01.169134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ba806-99ff-491b-af8d-39c7baa566e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:01.183392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c38ba806-99ff-491b-af8d-39c7baa566e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:01.319552 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9127ba-98be-4a8c-8268-fd9f97cc48d7 service cinder] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:01.353898 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5f5b5-b245-4739-ae4a-a3815697aaf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:01.363015 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5f5b5-b245-4739-ae4a-a3815697aaf2 service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:01.364999 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5f5b5-b245-4739-ae4a-a3815697aaf2 service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:01.366150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5f5b5-b245-4739-ae4a-a3815697aaf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:01.384209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ba806-99ff-491b-af8d-39c7baa566e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:01.388086 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ba806-99ff-491b-af8d-39c7baa566e7 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:01.395211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5f5b5-b245-4739-ae4a-a3815697aaf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:01.407602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5f5b5-b245-4739-ae4a-a3815697aaf2 service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:01.411190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 870/1746] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:01.477916 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 877/1747] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:13:01.864591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ba806-99ff-491b-af8d-39c7baa566e7 service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:01.867437 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c38ba806-99ff-491b-af8d-39c7baa566e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:01.954389 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:02.242382 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ba806-99ff-491b-af8d-39c7baa566e7 service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:02.268411 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85684e41-1f20-4135-91ce-93e0ff1ffb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:02.269509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85684e41-1f20-4135-91ce-93e0ff1ffb15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:02.269980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85684e41-1f20-4135-91ce-93e0ff1ffb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:02.270463 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85684e41-1f20-4135-91ce-93e0ff1ffb15 admin admin] PATH_INFO: `/v3/projects/e11f9598134c4de5b533cabdfc01a185` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:02.279494 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85684e41-1f20-4135-91ce-93e0ff1ffb15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e11f9598134c4de5b533cabdfc01a185)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:02.292031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85684e41-1f20-4135-91ce-93e0ff1ffb15 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:02.499617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff041c5-9177-4e93-9dd3-02f82f8b7d29 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:02.500397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff041c5-9177-4e93-9dd3-02f82f8b7d29 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:02.500397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff041c5-9177-4e93-9dd3-02f82f8b7d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:02.551074 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-85684e41-1f20-4135-91ce-93e0ff1ffb15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e11f9598134c4de5b533cabdfc01a185', 'request_id': 'req-85684e41-1f20-4135-91ce-93e0ff1ffb15'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:02.590321 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-85684e41-1f20-4135-91ce-93e0ff1ffb15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e11f9598134c4de5b533cabdfc01a185', 'request_id': 'req-85684e41-1f20-4135-91ce-93e0ff1ffb15'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:02.623908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-85684e41-1f20-4135-91ce-93e0ff1ffb15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e11f9598134c4de5b533cabdfc01a185', 'request_id': 'req-85684e41-1f20-4135-91ce-93e0ff1ffb15'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:02.670953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-85684e41-1f20-4135-91ce-93e0ff1ffb15 admin admin] The token cache is being invalidate because project e11f9598134c4de5b533cabdfc01a185 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:13:02.671879 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-85684e41-1f20-4135-91ce-93e0ff1ffb15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:02.673050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-aff041c5-9177-4e93-9dd3-02f82f8b7d29 None None] MFA Rules not processed for user `3af3a5c1c2f94e51bee936b4762dc14c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:02.675258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-85684e41-1f20-4135-91ce-93e0ff1ffb15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:02.677093 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-aff041c5-9177-4e93-9dd3-02f82f8b7d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:02.680167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 871/1748] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:13:01 2022] DELETE /identity/v3/projects/e11f9598134c4de5b533cabdfc01a185 => generated 0 bytes in 817 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:02.715668 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95fc29-1958-433e-a3b0-b1dae1b780ff None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:02.721921 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95fc29-1958-433e-a3b0-b1dae1b780ff None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:02.726116 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95fc29-1958-433e-a3b0-b1dae1b780ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:02.830391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:02.863879 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-4a95fc29-1958-433e-a3b0-b1dae1b780ff None None] MFA Rules not processed for user `5a49850f6d8a474d83821f5f49dff06c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:02.869088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-4a95fc29-1958-433e-a3b0-b1dae1b780ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:03.091238 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 878/1749] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 601 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:13:03.094711 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff041c5-9177-4e93-9dd3-02f82f8b7d29 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:03.101369 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-aff041c5-9177-4e93-9dd3-02f82f8b7d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:03.181551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 872/1750] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 501 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:03.188826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a962fa-13c0-49e8-ba66-bbbb569e2c00 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:03.189506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a962fa-13c0-49e8-ba66-bbbb569e2c00 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:03.190026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a962fa-13c0-49e8-ba66-bbbb569e2c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:03.257946 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-c2a962fa-13c0-49e8-ba66-bbbb569e2c00 None None] MFA Rules not processed for user `3af3a5c1c2f94e51bee936b4762dc14c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:03.264611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c2a962fa-13c0-49e8-ba66-bbbb569e2c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:03.296439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 873/1751] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:03.298450 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a962fa-13c0-49e8-ba66-bbbb569e2c00 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:03.317670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c2a962fa-13c0-49e8-ba66-bbbb569e2c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:03.327169 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff041c5-9177-4e93-9dd3-02f82f8b7d29 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:03.335763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07feb6a-02d8-4d98-ba6c-b34d3b3648ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:03.337170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07feb6a-02d8-4d98-ba6c-b34d3b3648ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:03.337686 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07feb6a-02d8-4d98-ba6c-b34d3b3648ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:03.341046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07feb6a-02d8-4d98-ba6c-b34d3b3648ee admin admin] PATH_INFO: `/v3/users/aa6781c391984c8a9f76de2b43739b8a` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:03.343226 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07feb6a-02d8-4d98-ba6c-b34d3b3648ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa6781c391984c8a9f76de2b43739b8a)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:03.348034 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07feb6a-02d8-4d98-ba6c-b34d3b3648ee admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:03.368593 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a962fa-13c0-49e8-ba66-bbbb569e2c00 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:03.389872 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad8b61-58dd-4f6c-977d-0b75453b1ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:03.390716 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad8b61-58dd-4f6c-977d-0b75453b1ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:03.391015 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad8b61-58dd-4f6c-977d-0b75453b1ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:03.391298 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad8b61-58dd-4f6c-977d-0b75453b1ee6 admin admin] PATH_INFO: `/v3/users/2aa9621d66a94553b363428cf3130249` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:03.394012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad8b61-58dd-4f6c-977d-0b75453b1ee6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aa9621d66a94553b363428cf3130249)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:03.399567 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad8b61-58dd-4f6c-977d-0b75453b1ee6 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:03.480727 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-c07feb6a-02d8-4d98-ba6c-b34d3b3648ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa6781c391984c8a9f76de2b43739b8a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:03.503792 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-c07feb6a-02d8-4d98-ba6c-b34d3b3648ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa6781c391984c8a9f76de2b43739b8a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:03.518744 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:03.538816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-c07feb6a-02d8-4d98-ba6c-b34d3b3648ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa6781c391984c8a9f76de2b43739b8a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:03.543003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-c07feb6a-02d8-4d98-ba6c-b34d3b3648ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa6781c391984c8a9f76de2b43739b8a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:03.544103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-c07feb6a-02d8-4d98-ba6c-b34d3b3648ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa6781c391984c8a9f76de2b43739b8a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:03.589378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a0ad8b61-58dd-4f6c-977d-0b75453b1ee6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aa9621d66a94553b363428cf3130249'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:03.590610 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a0ad8b61-58dd-4f6c-977d-0b75453b1ee6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aa9621d66a94553b363428cf3130249'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:03.624102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a0ad8b61-58dd-4f6c-977d-0b75453b1ee6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aa9621d66a94553b363428cf3130249'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:03.627263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a0ad8b61-58dd-4f6c-977d-0b75453b1ee6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aa9621d66a94553b363428cf3130249'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:03.643417 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 879/1752] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:13:03 2022] DELETE /identity/v3/users/aa6781c391984c8a9f76de2b43739b8a => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:03.645897 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07feb6a-02d8-4d98-ba6c-b34d3b3648ee admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:03.652150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-c07feb6a-02d8-4d98-ba6c-b34d3b3648ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:03.706679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a0ad8b61-58dd-4f6c-977d-0b75453b1ee6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aa9621d66a94553b363428cf3130249'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:03.775233 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 874/1753] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:13:03 2022] DELETE /identity/v3/users/2aa9621d66a94553b363428cf3130249 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:03.781737 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad8b61-58dd-4f6c-977d-0b75453b1ee6 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:03.818489 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a0ad8b61-58dd-4f6c-977d-0b75453b1ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:04.047467 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:04.143307 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07feb6a-02d8-4d98-ba6c-b34d3b3648ee admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:04.298149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad8b61-58dd-4f6c-977d-0b75453b1ee6 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:04.327845 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcc703-70d7-4d1d-8276-1376802e4b89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:04.329403 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcc703-70d7-4d1d-8276-1376802e4b89 service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:04.334022 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcc703-70d7-4d1d-8276-1376802e4b89 service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:04.334663 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcc703-70d7-4d1d-8276-1376802e4b89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:04.338298 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-27bcc703-70d7-4d1d-8276-1376802e4b89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:04.489077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e4b27-fbea-4df4-b6f5-b4c3a9d467d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:04.490614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e4b27-fbea-4df4-b6f5-b4c3a9d467d5 service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:04.491310 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e4b27-fbea-4df4-b6f5-b4c3a9d467d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:04.491935 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e4b27-fbea-4df4-b6f5-b4c3a9d467d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:04.500761 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e91e4b27-fbea-4df4-b6f5-b4c3a9d467d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:04.596488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bcc703-70d7-4d1d-8276-1376802e4b89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:04.611875 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bcc703-70d7-4d1d-8276-1376802e4b89 service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:04.685075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 875/1754] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 909 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:04.687624 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78578ad-e3b5-4d03-a8c2-ed554bfae797 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:04.696431 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78578ad-e3b5-4d03-a8c2-ed554bfae797 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:04.699868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78578ad-e3b5-4d03-a8c2-ed554bfae797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:04.740389 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e4b27-fbea-4df4-b6f5-b4c3a9d467d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:04.763040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e4b27-fbea-4df4-b6f5-b4c3a9d467d5 service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:04.803627 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 880/1755] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 1159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:04.854923 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-e78578ad-e3b5-4d03-a8c2-ed554bfae797 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:04.857246 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e78578ad-e3b5-4d03-a8c2-ed554bfae797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:05.039395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:05.175525 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 876/1756] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 490 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:05.236409 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e4b27-fbea-4df4-b6f5-b4c3a9d467d5 service nova] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:05.239114 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e91e4b27-fbea-4df4-b6f5-b4c3a9d467d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:05.422932 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78578ad-e3b5-4d03-a8c2-ed554bfae797 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:05.425502 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e78578ad-e3b5-4d03-a8c2-ed554bfae797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:05.453321 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78578ad-e3b5-4d03-a8c2-ed554bfae797 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:05.474677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb94502-e519-4495-8de1-70c292a60d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:05.475708 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb94502-e519-4495-8de1-70c292a60d8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:05.476983 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb94502-e519-4495-8de1-70c292a60d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:05.477567 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb94502-e519-4495-8de1-70c292a60d8c admin admin] PATH_INFO: `/v3/projects/ef208ca2fedf4f91812d06e8e90fb29f` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:05.479250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb94502-e519-4495-8de1-70c292a60d8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef208ca2fedf4f91812d06e8e90fb29f)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:05.485479 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb94502-e519-4495-8de1-70c292a60d8c admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:05.521204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e4b27-fbea-4df4-b6f5-b4c3a9d467d5 service nova] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:05.552932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ae825-75a9-4e71-ad56-22389c63c916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:05.556312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ae825-75a9-4e71-ad56-22389c63c916 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:05.556677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ae825-75a9-4e71-ad56-22389c63c916 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:05.566163 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ae825-75a9-4e71-ad56-22389c63c916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:05.628526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ae825-75a9-4e71-ad56-22389c63c916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:05.639733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ae825-75a9-4e71-ad56-22389c63c916 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:05.707820 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 881/1757] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:13:05.727112 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9cb94502-e519-4495-8de1-70c292a60d8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef208ca2fedf4f91812d06e8e90fb29f', 'request_id': 'req-9cb94502-e519-4495-8de1-70c292a60d8c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:05.753204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9cb94502-e519-4495-8de1-70c292a60d8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef208ca2fedf4f91812d06e8e90fb29f', 'request_id': 'req-9cb94502-e519-4495-8de1-70c292a60d8c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:05.784990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9cb94502-e519-4495-8de1-70c292a60d8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef208ca2fedf4f91812d06e8e90fb29f', 'request_id': 'req-9cb94502-e519-4495-8de1-70c292a60d8c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:05.823911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9cb94502-e519-4495-8de1-70c292a60d8c admin admin] The token cache is being invalidate because project ef208ca2fedf4f91812d06e8e90fb29f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:13:05.825033 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9cb94502-e519-4495-8de1-70c292a60d8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:05.839106 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9cb94502-e519-4495-8de1-70c292a60d8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:05.848806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 877/1758] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:13:05 2022] DELETE /identity/v3/projects/ef208ca2fedf4f91812d06e8e90fb29f => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:05.849898 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317029d3-4f33-473b-8c19-0002a612bbd1 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:05.850611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317029d3-4f33-473b-8c19-0002a612bbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:05.850611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317029d3-4f33-473b-8c19-0002a612bbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:05.993639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-317029d3-4f33-473b-8c19-0002a612bbd1 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:06.009657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-317029d3-4f33-473b-8c19-0002a612bbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:06.142810 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:06.403214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 882/1759] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 555 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:06.419317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62281ea-1196-4c3a-95e3-b8d6ef5c383e None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:06.419858 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62281ea-1196-4c3a-95e3-b8d6ef5c383e None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:06.420331 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62281ea-1196-4c3a-95e3-b8d6ef5c383e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:06.549368 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-e62281ea-1196-4c3a-95e3-b8d6ef5c383e None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:06.558725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e62281ea-1196-4c3a-95e3-b8d6ef5c383e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:06.694565 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317029d3-4f33-473b-8c19-0002a612bbd1 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:06.702740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 878/1760] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:06.740119 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317029d3-4f33-473b-8c19-0002a612bbd1 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:06.793405 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557e3e3-22db-463c-a74b-50c5d40223cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:06.800201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557e3e3-22db-463c-a74b-50c5d40223cf admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:06.802983 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557e3e3-22db-463c-a74b-50c5d40223cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:06.809691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557e3e3-22db-463c-a74b-50c5d40223cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:06.813671 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557e3e3-22db-463c-a74b-50c5d40223cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:06.815673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557e3e3-22db-463c-a74b-50c5d40223cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:06.844667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557e3e3-22db-463c-a74b-50c5d40223cf admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:06.918667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62281ea-1196-4c3a-95e3-b8d6ef5c383e None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:06.921075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 883/1761] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:06.936362 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62281ea-1196-4c3a-95e3-b8d6ef5c383e None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:06.960327 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b12da-3a56-4360-8fe1-bff9ec3894ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:06.961989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b12da-3a56-4360-8fe1-bff9ec3894ab admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:06.962462 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b12da-3a56-4360-8fe1-bff9ec3894ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:06.963118 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b12da-3a56-4360-8fe1-bff9ec3894ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:06.964413 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b12da-3a56-4360-8fe1-bff9ec3894ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:06.967337 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b12da-3a56-4360-8fe1-bff9ec3894ab admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:07.091424 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 879/1762] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:13:06 2022] POST /identity/v3/projects => generated 381 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:13:07.098496 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557e3e3-22db-463c-a74b-50c5d40223cf admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:07.134387 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557e3e3-22db-463c-a74b-50c5d40223cf admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:07.499247 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e58e98-39b8-4306-ad22-24ad79de6bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:07.501295 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e58e98-39b8-4306-ad22-24ad79de6bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:07.501710 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e58e98-39b8-4306-ad22-24ad79de6bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:07.502097 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e58e98-39b8-4306-ad22-24ad79de6bf2 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:07.513868 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e58e98-39b8-4306-ad22-24ad79de6bf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:07.519151 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e58e98-39b8-4306-ad22-24ad79de6bf2 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:07.608585 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 884/1763] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:13:07 2022] POST /identity/v3/users => generated 291 bytes in 513 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:13:07.616116 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b12da-3a56-4360-8fe1-bff9ec3894ab admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:07.648210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b12da-3a56-4360-8fe1-bff9ec3894ab admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:07.682318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f3620-4a4b-4386-894f-6b9fad1091f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:07.684248 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f3620-4a4b-4386-894f-6b9fad1091f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:07.684855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f3620-4a4b-4386-894f-6b9fad1091f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:07.685438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f3620-4a4b-4386-894f-6b9fad1091f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:07.686545 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f3620-4a4b-4386-894f-6b9fad1091f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:07.691783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f3620-4a4b-4386-894f-6b9fad1091f2 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:07.758797 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 880/1764] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:13:07.763541 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e58e98-39b8-4306-ad22-24ad79de6bf2 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:07.798320 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e58e98-39b8-4306-ad22-24ad79de6bf2 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:07.830833 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dcd5ba-e0c0-4370-a5f6-ec5410762951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:07.831909 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcd5ba-e0c0-4370-a5f6-ec5410762951 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:07.834553 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcd5ba-e0c0-4370-a5f6-ec5410762951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:07.835739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcd5ba-e0c0-4370-a5f6-ec5410762951 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:07.838822 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dcd5ba-e0c0-4370-a5f6-ec5410762951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:07.847872 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dcd5ba-e0c0-4370-a5f6-ec5410762951 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:07.904744 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 885/1765] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:13:07.911171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f3620-4a4b-4386-894f-6b9fad1091f2 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:07.934404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f3620-4a4b-4386-894f-6b9fad1091f2 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:07.957526 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f9e23-e5a1-4832-b1ff-2f6324aa2c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:07.958405 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f9e23-e5a1-4832-b1ff-2f6324aa2c9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:07.960435 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f9e23-e5a1-4832-b1ff-2f6324aa2c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:07.960773 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f9e23-e5a1-4832-b1ff-2f6324aa2c9e admin admin] PATH_INFO: `/v3/projects/75198ce09a0747a9b77affbe1a66d891/users/9183cb952fa94bf6bbcf2771f11cd66e/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:08.024667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f9e23-e5a1-4832-b1ff-2f6324aa2c9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=75198ce09a0747a9b77affbe1a66d891, user_id=9183cb952fa94bf6bbcf2771f11cd66e, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:08.029612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f9e23-e5a1-4832-b1ff-2f6324aa2c9e admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:08.067204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dcd5ba-e0c0-4370-a5f6-ec5410762951 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:08.071338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b8dcd5ba-e0c0-4370-a5f6-ec5410762951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:08.097858 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 881/1766] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:13:07 2022] PUT /identity/v3/projects/75198ce09a0747a9b77affbe1a66d891/users/9183cb952fa94bf6bbcf2771f11cd66e/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:08.107754 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9437d1-4e70-4bab-ae44-0300bce70d47 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:08.108666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9437d1-4e70-4bab-ae44-0300bce70d47 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:08.109742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9437d1-4e70-4bab-ae44-0300bce70d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:08.121069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dcd5ba-e0c0-4370-a5f6-ec5410762951 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:08.282921 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-ad9437d1-4e70-4bab-ae44-0300bce70d47 None None] MFA Rules not processed for user `9183cb952fa94bf6bbcf2771f11cd66e`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:08.290715 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ad9437d1-4e70-4bab-ae44-0300bce70d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:08.506955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:08.535785 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3c87b-2277-45c1-8732-901fee4a5c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:08.539382 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3c87b-2277-45c1-8732-901fee4a5c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:08.549519 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3c87b-2277-45c1-8732-901fee4a5c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:08.550431 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3c87b-2277-45c1-8732-901fee4a5c81 admin admin] PATH_INFO: `/v3/projects/624eecda61944ff1a74a880ace535a77` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:08.553306 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3c87b-2277-45c1-8732-901fee4a5c81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=624eecda61944ff1a74a880ace535a77)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:08.562905 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3c87b-2277-45c1-8732-901fee4a5c81 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:08.617824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 882/1767] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 513 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:08.623685 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49f218-2c7d-403f-8877-a8a0bd49a307 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:08.624744 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49f218-2c7d-403f-8877-a8a0bd49a307 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:08.626471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49f218-2c7d-403f-8877-a8a0bd49a307 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:08.643266 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 883/1768] 10.222.0.93 () {54 vars in 912 bytes} [Mon Aug 1 15:13:08 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:13:08.661166 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49f218-2c7d-403f-8877-a8a0bd49a307 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:08.720376 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49f218-2c7d-403f-8877-a8a0bd49a307 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:08.737062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3fe9c-53bb-4e54-bdb4-288eef6d8299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:08.737902 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3fe9c-53bb-4e54-bdb4-288eef6d8299 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:08.738348 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3fe9c-53bb-4e54-bdb4-288eef6d8299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:08.738825 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3fe9c-53bb-4e54-bdb4-288eef6d8299 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:08.740035 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3fe9c-53bb-4e54-bdb4-288eef6d8299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:08.755650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3fe9c-53bb-4e54-bdb4-288eef6d8299 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:08.762427 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:08.847218 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-17b3c87b-2277-45c1-8732-901fee4a5c81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '624eecda61944ff1a74a880ace535a77', 'request_id': 'req-17b3c87b-2277-45c1-8732-901fee4a5c81'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:08.857790 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 884/1769] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:13:08 2022] POST /identity/v3/projects => generated 379 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:13:08.860859 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aedaa-241d-42a9-9abe-cd9ae4c804cf None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:08.867155 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aedaa-241d-42a9-9abe-cd9ae4c804cf None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:08.867697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aedaa-241d-42a9-9abe-cd9ae4c804cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:08.883501 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-17b3c87b-2277-45c1-8732-901fee4a5c81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '624eecda61944ff1a74a880ace535a77', 'request_id': 'req-17b3c87b-2277-45c1-8732-901fee4a5c81'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:08.947368 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-17b3c87b-2277-45c1-8732-901fee4a5c81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '624eecda61944ff1a74a880ace535a77', 'request_id': 'req-17b3c87b-2277-45c1-8732-901fee4a5c81'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:08.999677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-034aedaa-241d-42a9-9abe-cd9ae4c804cf None None] MFA Rules not processed for user `4eba5f64833b482d81c53721c8442970`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:09.002481 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-034aedaa-241d-42a9-9abe-cd9ae4c804cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:09.070029 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-17b3c87b-2277-45c1-8732-901fee4a5c81 admin admin] The token cache is being invalidate because project 624eecda61944ff1a74a880ace535a77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:13:09.074471 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-17b3c87b-2277-45c1-8732-901fee4a5c81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:09.077596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-17b3c87b-2277-45c1-8732-901fee4a5c81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:09.093833 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 886/1770] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:13:08 2022] DELETE /identity/v3/projects/624eecda61944ff1a74a880ace535a77 => generated 0 bytes in 1022 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:13:09.101479 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3c87b-2277-45c1-8732-901fee4a5c81 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:09.110840 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-17b3c87b-2277-45c1-8732-901fee4a5c81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:09.321806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 885/1771] 10.222.0.93 () {58 vars in 1009 bytes} [Mon Aug 1 15:13:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:09.330416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aedaa-241d-42a9-9abe-cd9ae4c804cf None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:09.341139 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-034aedaa-241d-42a9-9abe-cd9ae4c804cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:09.520230 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3c87b-2277-45c1-8732-901fee4a5c81 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:09.665307 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aedaa-241d-42a9-9abe-cd9ae4c804cf None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:09.701159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783eedbd-7d57-4a91-925f-1aca63c0ff88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:09.703647 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783eedbd-7d57-4a91-925f-1aca63c0ff88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:09.706144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783eedbd-7d57-4a91-925f-1aca63c0ff88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:09.706676 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783eedbd-7d57-4a91-925f-1aca63c0ff88 admin admin] PATH_INFO: `/v3/users/46bbc8aecaab4abebf0db6aae42a834b` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:09.714550 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783eedbd-7d57-4a91-925f-1aca63c0ff88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46bbc8aecaab4abebf0db6aae42a834b)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:09.720518 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783eedbd-7d57-4a91-925f-1aca63c0ff88 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:09.834406 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b526f-eb9b-4df8-a5cf-920186b823b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:09.837968 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b526f-eb9b-4df8-a5cf-920186b823b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:09.838495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b526f-eb9b-4df8-a5cf-920186b823b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:09.846101 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b526f-eb9b-4df8-a5cf-920186b823b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:09.852377 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b526f-eb9b-4df8-a5cf-920186b823b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:09.869284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b526f-eb9b-4df8-a5cf-920186b823b5 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:09.923898 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-783eedbd-7d57-4a91-925f-1aca63c0ff88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46bbc8aecaab4abebf0db6aae42a834b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:09.929226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-783eedbd-7d57-4a91-925f-1aca63c0ff88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46bbc8aecaab4abebf0db6aae42a834b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:09.962641 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 887/1772] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:13:09 2022] POST /identity/v3/users => generated 296 bytes in 866 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:13:09.964143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b526f-eb9b-4df8-a5cf-920186b823b5 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:09.974192 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-335b526f-eb9b-4df8-a5cf-920186b823b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:09.979754 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-783eedbd-7d57-4a91-925f-1aca63c0ff88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46bbc8aecaab4abebf0db6aae42a834b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:09.981471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-783eedbd-7d57-4a91-925f-1aca63c0ff88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46bbc8aecaab4abebf0db6aae42a834b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:10.024389 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:10.052777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b526f-eb9b-4df8-a5cf-920186b823b5 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:10.054979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-783eedbd-7d57-4a91-925f-1aca63c0ff88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46bbc8aecaab4abebf0db6aae42a834b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:10.069473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579da1ca-d22a-44fd-b772-e03101122fa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:10.072454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579da1ca-d22a-44fd-b772-e03101122fa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:10.074035 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579da1ca-d22a-44fd-b772-e03101122fa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:10.074035 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579da1ca-d22a-44fd-b772-e03101122fa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:10.092658 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-579da1ca-d22a-44fd-b772-e03101122fa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:10.128436 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 886/1773] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:13:09 2022] DELETE /identity/v3/users/46bbc8aecaab4abebf0db6aae42a834b => generated 0 bytes in 806 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:10.129871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783eedbd-7d57-4a91-925f-1aca63c0ff88 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:10.134739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-783eedbd-7d57-4a91-925f-1aca63c0ff88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:10.154030 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579da1ca-d22a-44fd-b772-e03101122fa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:10.155553 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579da1ca-d22a-44fd-b772-e03101122fa5 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:10.198149 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 888/1774] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:10.221785 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579da1ca-d22a-44fd-b772-e03101122fa5 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:10.264289 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-579da1ca-d22a-44fd-b772-e03101122fa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:10.375340 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579da1ca-d22a-44fd-b772-e03101122fa5 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:10.528977 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:10.547372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783eedbd-7d57-4a91-925f-1aca63c0ff88 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:10.557130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025d6f7-f212-4ebe-9231-c530e13218d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:10.558744 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025d6f7-f212-4ebe-9231-c530e13218d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:10.559492 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025d6f7-f212-4ebe-9231-c530e13218d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:10.560127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025d6f7-f212-4ebe-9231-c530e13218d6 admin admin] PATH_INFO: `/v3/users/d802c9d3c96d4009aefbfdde9c365543` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:10.563184 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9025d6f7-f212-4ebe-9231-c530e13218d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d802c9d3c96d4009aefbfdde9c365543)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:10.572790 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9025d6f7-f212-4ebe-9231-c530e13218d6 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:10.638174 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f42c0-e079-4939-935e-a8eef027c949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:10.641852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f42c0-e079-4939-935e-a8eef027c949 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:10.653147 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f42c0-e079-4939-935e-a8eef027c949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:10.655627 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f42c0-e079-4939-935e-a8eef027c949 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:10.658572 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f42c0-e079-4939-935e-a8eef027c949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:10.668516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f42c0-e079-4939-935e-a8eef027c949 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:10.744371 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 889/1775] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 543 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:13:10.748392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f42c0-e079-4939-935e-a8eef027c949 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:10.803463 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f42c0-e079-4939-935e-a8eef027c949 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:10.821199 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194e53e-7627-4cfa-ad6f-65b0ad8bebf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:10.827222 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194e53e-7627-4cfa-ad6f-65b0ad8bebf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:10.831049 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194e53e-7627-4cfa-ad6f-65b0ad8bebf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:10.831812 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194e53e-7627-4cfa-ad6f-65b0ad8bebf3 admin admin] PATH_INFO: `/v3/projects/23fb442a32bd4cf2999b1ca6a3f921ed/users/b9425868e7534a4e9c7d69a5ec7255a3/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:10.839895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9025d6f7-f212-4ebe-9231-c530e13218d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd802c9d3c96d4009aefbfdde9c365543'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:10.843513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9025d6f7-f212-4ebe-9231-c530e13218d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd802c9d3c96d4009aefbfdde9c365543'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:10.906928 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9025d6f7-f212-4ebe-9231-c530e13218d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd802c9d3c96d4009aefbfdde9c365543'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:10.908437 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9025d6f7-f212-4ebe-9231-c530e13218d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd802c9d3c96d4009aefbfdde9c365543'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:10.911964 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194e53e-7627-4cfa-ad6f-65b0ad8bebf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23fb442a32bd4cf2999b1ca6a3f921ed, user_id=b9425868e7534a4e9c7d69a5ec7255a3, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:10.919990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194e53e-7627-4cfa-ad6f-65b0ad8bebf3 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:10.974845 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 890/1776] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:13:10 2022] PUT /identity/v3/projects/23fb442a32bd4cf2999b1ca6a3f921ed/users/b9425868e7534a4e9c7d69a5ec7255a3/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:10.979631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194e53e-7627-4cfa-ad6f-65b0ad8bebf3 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:10.980593 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9025d6f7-f212-4ebe-9231-c530e13218d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd802c9d3c96d4009aefbfdde9c365543'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:10.987287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4194e53e-7627-4cfa-ad6f-65b0ad8bebf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:11.065597 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 887/1777] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:13:10 2022] DELETE /identity/v3/users/d802c9d3c96d4009aefbfdde9c365543 => generated 0 bytes in 936 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:11.072700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bf89d-2165-4498-a971-20861379cb0a None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:11.075758 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bf89d-2165-4498-a971-20861379cb0a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:11.079596 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bf89d-2165-4498-a971-20861379cb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:11.174755 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-ae2bf89d-2165-4498-a971-20861379cb0a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:11.183851 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ae2bf89d-2165-4498-a971-20861379cb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:11.257326 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194e53e-7627-4cfa-ad6f-65b0ad8bebf3 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:11.518084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01d0b1-6a4d-4063-851b-dd185fa90160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:11.518891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01d0b1-6a4d-4063-851b-dd185fa90160 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:11.522320 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01d0b1-6a4d-4063-851b-dd185fa90160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:11.525010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01d0b1-6a4d-4063-851b-dd185fa90160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:11.529212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01d0b1-6a4d-4063-851b-dd185fa90160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:11.543123 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01d0b1-6a4d-4063-851b-dd185fa90160 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:11.546363 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 888/1778] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:11.586018 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 891/1779] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 610 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:13:11.587023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bf89d-2165-4498-a971-20861379cb0a None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:11.590170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01d0b1-6a4d-4063-851b-dd185fa90160 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:11.594423 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0b01d0b1-6a4d-4063-851b-dd185fa90160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:11.599462 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:11.636655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bf89d-2165-4498-a971-20861379cb0a None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:11.660896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163bf53-df9e-48ed-ac5f-bf60556242d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:11.665426 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163bf53-df9e-48ed-ac5f-bf60556242d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:11.665984 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163bf53-df9e-48ed-ac5f-bf60556242d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:11.666447 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163bf53-df9e-48ed-ac5f-bf60556242d1 admin admin] PATH_INFO: `/v3/projects/23fb442a32bd4cf2999b1ca6a3f921ed/users/b9425868e7534a4e9c7d69a5ec7255a3/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:11.673503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163bf53-df9e-48ed-ac5f-bf60556242d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23fb442a32bd4cf2999b1ca6a3f921ed, user_id=b9425868e7534a4e9c7d69a5ec7255a3, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:11.688272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163bf53-df9e-48ed-ac5f-bf60556242d1 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:11.736176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 889/1780] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:13:11 2022] PUT /identity/v3/projects/23fb442a32bd4cf2999b1ca6a3f921ed/users/b9425868e7534a4e9c7d69a5ec7255a3/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:11.758295 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eeb868-747d-495e-b2fe-10088b555ba6 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:11.758798 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eeb868-747d-495e-b2fe-10088b555ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:11.759294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eeb868-747d-495e-b2fe-10088b555ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:11.857203 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-55eeb868-747d-495e-b2fe-10088b555ba6 None None] MFA Rules not processed for user `b9425868e7534a4e9c7d69a5ec7255a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:11.866959 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-55eeb868-747d-495e-b2fe-10088b555ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:11.926785 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01d0b1-6a4d-4063-851b-dd185fa90160 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:11.968838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:12.112390 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f1e7c2-4d37-49ed-bb54-c433ebf63ede service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:12.118091 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1e7c2-4d37-49ed-bb54-c433ebf63ede service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:12.118902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1e7c2-4d37-49ed-bb54-c433ebf63ede service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:12.120819 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1e7c2-4d37-49ed-bb54-c433ebf63ede service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:12.226613 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 890/1781] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:12.240999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eeb868-747d-495e-b2fe-10088b555ba6 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:12.243104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-55eeb868-747d-495e-b2fe-10088b555ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:12.373193 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f1e7c2-4d37-49ed-bb54-c433ebf63ede service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:12.379522 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f1e7c2-4d37-49ed-bb54-c433ebf63ede service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:12.420551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 892/1782] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 833 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:13:12.524908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eeb868-747d-495e-b2fe-10088b555ba6 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:12.540041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8691b-1754-4de9-9842-6976ac2d6552 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:12.543577 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8691b-1754-4de9-9842-6976ac2d6552 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:12.544064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8691b-1754-4de9-9842-6976ac2d6552 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:12.545664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8691b-1754-4de9-9842-6976ac2d6552 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:12.613791 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8691b-1754-4de9-9842-6976ac2d6552 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:12.618301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8691b-1754-4de9-9842-6976ac2d6552 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:12.674910 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9b7f4-ac7b-4e8b-8419-5bfc8587c7f7 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:12.675606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9b7f4-ac7b-4e8b-8419-5bfc8587c7f7 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:12.676237 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9b7f4-ac7b-4e8b-8419-5bfc8587c7f7 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:12.678064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 893/1784] 10.222.0.93 () {54 vars in 961 bytes} [Mon Aug 1 15:13:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:13:12.678366 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 891/1784] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:12.731333 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262a936-939c-4e9c-866b-70507e78c7ec None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:12.732517 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262a936-939c-4e9c-866b-70507e78c7ec None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:12.733299 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262a936-939c-4e9c-866b-70507e78c7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:12.817956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-6262a936-939c-4e9c-866b-70507e78c7ec None None] MFA Rules not processed for user `a07e3e25e6a8466fb237dcb836aecda2`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:12.824258 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-6262a936-939c-4e9c-866b-70507e78c7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:12.884427 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981671db-1c72-4f5f-8a13-906dba86de9d None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:12.885166 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981671db-1c72-4f5f-8a13-906dba86de9d None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:12.885971 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981671db-1c72-4f5f-8a13-906dba86de9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:13.016134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-981671db-1c72-4f5f-8a13-906dba86de9d None None] MFA Rules not processed for user `e708da2136a248c196ae4ecb531bb322`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:13.018058 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-981671db-1c72-4f5f-8a13-906dba86de9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:13.251926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 894/1785] 10.222.0.93 () {58 vars in 1058 bytes} [Mon Aug 1 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 523 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:13.254921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687495cc-1510-4d92-a9dd-cd63e814557e None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:13.266389 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687495cc-1510-4d92-a9dd-cd63e814557e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:13.266945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687495cc-1510-4d92-a9dd-cd63e814557e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:13.374391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-687495cc-1510-4d92-a9dd-cd63e814557e None None] MFA Rules not processed for user `3af3a5c1c2f94e51bee936b4762dc14c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:13.384348 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-687495cc-1510-4d92-a9dd-cd63e814557e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:13.513402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 892/1786] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 619 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:13.518650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981671db-1c72-4f5f-8a13-906dba86de9d None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:13.540644 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981671db-1c72-4f5f-8a13-906dba86de9d None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:13.558135 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8d751-2a49-40e9-9040-0bb02cba0b6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07e3e25e6a8466fb237dcb836aecda2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:13.559155 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8d751-2a49-40e9-9040-0bb02cba0b6b service placement] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:13.559867 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8d751-2a49-40e9-9040-0bb02cba0b6b service placement] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:13.560664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8d751-2a49-40e9-9040-0bb02cba0b6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:13.564864 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-15f8d751-2a49-40e9-9040-0bb02cba0b6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:13.596639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:13.611484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f8d751-2a49-40e9-9040-0bb02cba0b6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:13.614065 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f8d751-2a49-40e9-9040-0bb02cba0b6b service placement] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:13.668232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 893/1787] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:13.678727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3559063-69ff-4ca2-9390-638dd5c0d184 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:13.679652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3559063-69ff-4ca2-9390-638dd5c0d184 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:13.680558 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3559063-69ff-4ca2-9390-638dd5c0d184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:13.765080 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-a3559063-69ff-4ca2-9390-638dd5c0d184 None None] MFA Rules not processed for user `3c18437883ce42369c7563224c208523`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:13.775943 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a3559063-69ff-4ca2-9390-638dd5c0d184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:13.781377 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 895/1788] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 528 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:13.794273 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c7fde-9021-4235-9deb-c236d4fcf285 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:13.794634 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c7fde-9021-4235-9deb-c236d4fcf285 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:13.796426 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c7fde-9021-4235-9deb-c236d4fcf285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:13.826251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:13.868079 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-083c7fde-9021-4235-9deb-c236d4fcf285 None None] MFA Rules not processed for user `e708da2136a248c196ae4ecb531bb322`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:13.871674 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-083c7fde-9021-4235-9deb-c236d4fcf285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:13.913756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 896/1789] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:13.923750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c7fde-9021-4235-9deb-c236d4fcf285 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:13.934693 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-083c7fde-9021-4235-9deb-c236d4fcf285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:13.983319 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c7fde-9021-4235-9deb-c236d4fcf285 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:14.000275 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854f8d8-23f4-4dbf-b5bd-cd39c05b4def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:14.001474 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854f8d8-23f4-4dbf-b5bd-cd39c05b4def admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:14.004949 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854f8d8-23f4-4dbf-b5bd-cd39c05b4def admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:14.006632 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854f8d8-23f4-4dbf-b5bd-cd39c05b4def admin admin] PATH_INFO: `/v3/projects/8b6014b2454d4e09ac4b5aa01a751222` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:14.011061 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854f8d8-23f4-4dbf-b5bd-cd39c05b4def admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b6014b2454d4e09ac4b5aa01a751222)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:14.017545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854f8d8-23f4-4dbf-b5bd-cd39c05b4def admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:14.081021 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 894/1790] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:14.104942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3559063-69ff-4ca2-9390-638dd5c0d184 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:14.115965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a3559063-69ff-4ca2-9390-638dd5c0d184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:14.269703 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-2854f8d8-23f4-4dbf-b5bd-cd39c05b4def admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b6014b2454d4e09ac4b5aa01a751222', 'request_id': 'req-2854f8d8-23f4-4dbf-b5bd-cd39c05b4def'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:14.296930 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-2854f8d8-23f4-4dbf-b5bd-cd39c05b4def admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b6014b2454d4e09ac4b5aa01a751222', 'request_id': 'req-2854f8d8-23f4-4dbf-b5bd-cd39c05b4def'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:14.314146 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-2854f8d8-23f4-4dbf-b5bd-cd39c05b4def admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b6014b2454d4e09ac4b5aa01a751222', 'request_id': 'req-2854f8d8-23f4-4dbf-b5bd-cd39c05b4def'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:14.381095 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-2854f8d8-23f4-4dbf-b5bd-cd39c05b4def admin admin] The token cache is being invalidate because project 8b6014b2454d4e09ac4b5aa01a751222 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:13:14.382049 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-2854f8d8-23f4-4dbf-b5bd-cd39c05b4def admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:14.386225 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-2854f8d8-23f4-4dbf-b5bd-cd39c05b4def admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:14.400621 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 897/1791] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:13:13 2022] DELETE /identity/v3/projects/8b6014b2454d4e09ac4b5aa01a751222 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:14.402587 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854f8d8-23f4-4dbf-b5bd-cd39c05b4def admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:14.409734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-2854f8d8-23f4-4dbf-b5bd-cd39c05b4def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:14.415754 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3559063-69ff-4ca2-9390-638dd5c0d184 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:14.730307 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854f8d8-23f4-4dbf-b5bd-cd39c05b4def admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:14.741447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa274650-f544-4540-bb88-9e9e17f34a17 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9425868e7534a4e9c7d69a5ec7255a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23fb442a32bd4cf2999b1ca6a3f921ed', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:14.746524 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa274650-f544-4540-bb88-9e9e17f34a17 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:14.747383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa274650-f544-4540-bb88-9e9e17f34a17 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:14.747728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa274650-f544-4540-bb88-9e9e17f34a17 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:14.749194 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa274650-f544-4540-bb88-9e9e17f34a17 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:14.749806 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa274650-f544-4540-bb88-9e9e17f34a17 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:14.753090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa274650-f544-4540-bb88-9e9e17f34a17 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:14.769459 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee731f29-fd99-432f-befb-4215c7b4e74a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:14.770765 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee731f29-fd99-432f-befb-4215c7b4e74a service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:14.771291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee731f29-fd99-432f-befb-4215c7b4e74a service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:14.774896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee731f29-fd99-432f-befb-4215c7b4e74a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:14.786103 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ee731f29-fd99-432f-befb-4215c7b4e74a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:14.818596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 898/1792] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:14.830581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68ae7f-5d5f-42d1-b30d-4832cf174f30 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:14.832066 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68ae7f-5d5f-42d1-b30d-4832cf174f30 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:14.833643 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68ae7f-5d5f-42d1-b30d-4832cf174f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:14.927181 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-be68ae7f-5d5f-42d1-b30d-4832cf174f30 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:14.936289 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-be68ae7f-5d5f-42d1-b30d-4832cf174f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:15.007777 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:15.136886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee731f29-fd99-432f-befb-4215c7b4e74a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:15.142690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee731f29-fd99-432f-befb-4215c7b4e74a service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:15.253544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 895/1793] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 1171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:15.256032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee731f29-fd99-432f-befb-4215c7b4e74a service nova] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:15.310677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee731f29-fd99-432f-befb-4215c7b4e74a service nova] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:15.324325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 899/1794] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 499 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:15.338403 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6f324-492a-45e4-bda3-dd96e632274b tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9425868e7534a4e9c7d69a5ec7255a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23fb442a32bd4cf2999b1ca6a3f921ed', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:15.339822 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6f324-492a-45e4-bda3-dd96e632274b tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:15.340369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6f324-492a-45e4-bda3-dd96e632274b tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:15.340695 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6f324-492a-45e4-bda3-dd96e632274b tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:15.343861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88db58-b817-4ad4-9a50-edd1a239a5fe None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:15.344360 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88db58-b817-4ad4-9a50-edd1a239a5fe None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:15.344915 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88db58-b817-4ad4-9a50-edd1a239a5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:15.355286 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b6f324-492a-45e4-bda3-dd96e632274b tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:15.374126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b6f324-492a-45e4-bda3-dd96e632274b tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:15.457954 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-cd88db58-b817-4ad4-9a50-edd1a239a5fe None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:15.463421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-cd88db58-b817-4ad4-9a50-edd1a239a5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:15.508233 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 896/1795] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:13:15 2022] POST /identity/v3/projects => generated 354 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:13:15.511943 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6f324-492a-45e4-bda3-dd96e632274b tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:15.516859 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 900/1796] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:13:15.520794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88db58-b817-4ad4-9a50-edd1a239a5fe None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:15.565002 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88db58-b817-4ad4-9a50-edd1a239a5fe None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:15.583248 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:15.846565 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66e488-caec-48c8-95d7-62b790a989b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:15.857963 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66e488-caec-48c8-95d7-62b790a989b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:15.858724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66e488-caec-48c8-95d7-62b790a989b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:15.859983 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66e488-caec-48c8-95d7-62b790a989b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:15.862426 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66e488-caec-48c8-95d7-62b790a989b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:15.864102 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66e488-caec-48c8-95d7-62b790a989b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:15.869865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66e488-caec-48c8-95d7-62b790a989b3 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:15.914880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 901/1797] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:15.924802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66e488-caec-48c8-95d7-62b790a989b3 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:15.976978 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6f324-492a-45e4-bda3-dd96e632274b tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:16.003318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f4f64-50a6-480a-b57d-89c8be82faba tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9425868e7534a4e9c7d69a5ec7255a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23fb442a32bd4cf2999b1ca6a3f921ed', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:16.005011 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f4f64-50a6-480a-b57d-89c8be82faba tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:16.005722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f4f64-50a6-480a-b57d-89c8be82faba tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:16.006408 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f4f64-50a6-480a-b57d-89c8be82faba tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:16.011560 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66e488-caec-48c8-95d7-62b790a989b3 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:16.012206 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f4f64-50a6-480a-b57d-89c8be82faba tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:16.012908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f4f64-50a6-480a-b57d-89c8be82faba tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:16.018771 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f4f64-50a6-480a-b57d-89c8be82faba tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:16.040789 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d45ef0-4ea9-41e1-b718-81b10b74f080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:16.045708 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d45ef0-4ea9-41e1-b718-81b10b74f080 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:16.046209 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d45ef0-4ea9-41e1-b718-81b10b74f080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:16.049269 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d45ef0-4ea9-41e1-b718-81b10b74f080 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:16.071430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d45ef0-4ea9-41e1-b718-81b10b74f080 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:16.077290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 897/1798] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 567 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:16.097635 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d45ef0-4ea9-41e1-b718-81b10b74f080 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:16.216948 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f4f64-50a6-480a-b57d-89c8be82faba tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:16.285399 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f4f64-50a6-480a-b57d-89c8be82faba tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:16.291324 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 902/1799] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:13:15 2022] POST /identity/v3/projects => generated 385 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:13:16.299862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d45ef0-4ea9-41e1-b718-81b10b74f080 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:16.611988 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e7fbe-c2c5-4df2-a0a7-6fbf54636d39 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9425868e7534a4e9c7d69a5ec7255a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23fb442a32bd4cf2999b1ca6a3f921ed', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:16.612707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e7fbe-c2c5-4df2-a0a7-6fbf54636d39 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:16.613038 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e7fbe-c2c5-4df2-a0a7-6fbf54636d39 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:16.613320 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e7fbe-c2c5-4df2-a0a7-6fbf54636d39 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] PATH_INFO: `/v3/projects/602764ad94c043b6ac0d7278efbbda80` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:16.621950 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e7fbe-c2c5-4df2-a0a7-6fbf54636d39 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] RBAC: Authorizing `identity:delete_project(project_id=602764ad94c043b6ac0d7278efbbda80)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:16.632383 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e7fbe-c2c5-4df2-a0a7-6fbf54636d39 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:16.647316 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d45ef0-4ea9-41e1-b718-81b10b74f080 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:16.673523 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03339c2-077c-48a7-9feb-c28006bc690f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:16.675342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03339c2-077c-48a7-9feb-c28006bc690f admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:16.675774 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03339c2-077c-48a7-9feb-c28006bc690f admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:16.676231 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03339c2-077c-48a7-9feb-c28006bc690f admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:16.679437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03339c2-077c-48a7-9feb-c28006bc690f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:16.687761 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03339c2-077c-48a7-9feb-c28006bc690f admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:16.760094 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 903/1800] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:13:16 2022] POST /identity/v3/users => generated 293 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:13:16.763297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03339c2-077c-48a7-9feb-c28006bc690f admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:16.802623 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03339c2-077c-48a7-9feb-c28006bc690f admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:16.808783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:16.816618 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-815e7fbe-c2c5-4df2-a0a7-6fbf54636d39 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '602764ad94c043b6ac0d7278efbbda80', 'request_id': 'req-815e7fbe-c2c5-4df2-a0a7-6fbf54636d39'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:16.867790 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-815e7fbe-c2c5-4df2-a0a7-6fbf54636d39 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '602764ad94c043b6ac0d7278efbbda80', 'request_id': 'req-815e7fbe-c2c5-4df2-a0a7-6fbf54636d39'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:16.909373 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-815e7fbe-c2c5-4df2-a0a7-6fbf54636d39 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '602764ad94c043b6ac0d7278efbbda80', 'request_id': 'req-815e7fbe-c2c5-4df2-a0a7-6fbf54636d39'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:16.948981 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-815e7fbe-c2c5-4df2-a0a7-6fbf54636d39 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] The token cache is being invalidate because project 602764ad94c043b6ac0d7278efbbda80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:13:16.952041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-815e7fbe-c2c5-4df2-a0a7-6fbf54636d39 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:16.954023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-815e7fbe-c2c5-4df2-a0a7-6fbf54636d39 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:16.959648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 898/1801] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:13:16 2022] DELETE /identity/v3/projects/602764ad94c043b6ac0d7278efbbda80 => generated 0 bytes in 743 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:13:17.084123 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae932c-54c3-4ca7-b8b2-a4cbb44ddf61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:17.087696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae932c-54c3-4ca7-b8b2-a4cbb44ddf61 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:17.087696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae932c-54c3-4ca7-b8b2-a4cbb44ddf61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:17.087696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae932c-54c3-4ca7-b8b2-a4cbb44ddf61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:17.087696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae932c-54c3-4ca7-b8b2-a4cbb44ddf61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:17.090865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae932c-54c3-4ca7-b8b2-a4cbb44ddf61 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:17.132103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 904/1802] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:13:17.137002 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e7fbe-c2c5-4df2-a0a7-6fbf54636d39 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:17.139020 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-815e7fbe-c2c5-4df2-a0a7-6fbf54636d39 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:17.209525 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e7fbe-c2c5-4df2-a0a7-6fbf54636d39 tempest-VolumeQuotaClassesTest-120828816 tempest-VolumeQuotaClassesTest-120828816-project-admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:17.235583 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f6e15-b473-4cbd-a8ab-652022981a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:17.242022 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f6e15-b473-4cbd-a8ab-652022981a5b admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:17.242492 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f6e15-b473-4cbd-a8ab-652022981a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:17.245368 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f6e15-b473-4cbd-a8ab-652022981a5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:17.251944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f6e15-b473-4cbd-a8ab-652022981a5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:17.263038 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f6e15-b473-4cbd-a8ab-652022981a5b admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:17.326419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 899/1803] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:13:17.333240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae932c-54c3-4ca7-b8b2-a4cbb44ddf61 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:17.351332 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:17.374725 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae932c-54c3-4ca7-b8b2-a4cbb44ddf61 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:17.414807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa0dbd-43c0-4c54-be7a-fe28149648e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:17.417134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa0dbd-43c0-4c54-be7a-fe28149648e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:17.418119 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa0dbd-43c0-4c54-be7a-fe28149648e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:17.418601 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f6e15-b473-4cbd-a8ab-652022981a5b admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:17.422191 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d61f6e15-b473-4cbd-a8ab-652022981a5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:17.427723 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa0dbd-43c0-4c54-be7a-fe28149648e0 admin admin] PATH_INFO: `/v3/projects/f0871b19488d4b23b37dc5e1e776c3da/users/69461e0bd4494c64b26331696ca0eb6a/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:17.476636 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f6e15-b473-4cbd-a8ab-652022981a5b admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:17.497264 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3e415-1aea-4a11-9c5f-38283776c717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:17.498198 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3e415-1aea-4a11-9c5f-38283776c717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:17.498592 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3e415-1aea-4a11-9c5f-38283776c717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:17.499003 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3e415-1aea-4a11-9c5f-38283776c717 admin admin] PATH_INFO: `/v3/projects/cf289e1122ba465082fd2f9b7f5d388a` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:17.505359 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3e415-1aea-4a11-9c5f-38283776c717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf289e1122ba465082fd2f9b7f5d388a)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:17.508377 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3e415-1aea-4a11-9c5f-38283776c717 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:17.513533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa0dbd-43c0-4c54-be7a-fe28149648e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0871b19488d4b23b37dc5e1e776c3da, user_id=69461e0bd4494c64b26331696ca0eb6a, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:17.526799 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa0dbd-43c0-4c54-be7a-fe28149648e0 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:17.567844 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 905/1804] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:13:17 2022] PUT /identity/v3/projects/f0871b19488d4b23b37dc5e1e776c3da/users/69461e0bd4494c64b26331696ca0eb6a/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:17.570585 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0391a-27fc-4234-a0f1-95e3ff998dba None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:17.571210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0391a-27fc-4234-a0f1-95e3ff998dba None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:17.572768 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0391a-27fc-4234-a0f1-95e3ff998dba None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:17.576613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 906/1805] 10.222.0.93 () {54 vars in 912 bytes} [Mon Aug 1 15:13:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:13:17.595251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51012d7b-e994-4211-a5ed-6fb5268865c2 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:17.595771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51012d7b-e994-4211-a5ed-6fb5268865c2 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:17.596304 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51012d7b-e994-4211-a5ed-6fb5268865c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:17.694313 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-51012d7b-e994-4211-a5ed-6fb5268865c2 None None] MFA Rules not processed for user `4eba5f64833b482d81c53721c8442970`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:17.695594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-51012d7b-e994-4211-a5ed-6fb5268865c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:17.721346 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4db3e415-1aea-4a11-9c5f-38283776c717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf289e1122ba465082fd2f9b7f5d388a', 'request_id': 'req-4db3e415-1aea-4a11-9c5f-38283776c717'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:17.736597 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4db3e415-1aea-4a11-9c5f-38283776c717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf289e1122ba465082fd2f9b7f5d388a', 'request_id': 'req-4db3e415-1aea-4a11-9c5f-38283776c717'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:17.752966 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4db3e415-1aea-4a11-9c5f-38283776c717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf289e1122ba465082fd2f9b7f5d388a', 'request_id': 'req-4db3e415-1aea-4a11-9c5f-38283776c717'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:17.826181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4db3e415-1aea-4a11-9c5f-38283776c717 admin admin] The token cache is being invalidate because project cf289e1122ba465082fd2f9b7f5d388a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:13:17.827012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4db3e415-1aea-4a11-9c5f-38283776c717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:17.831126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4db3e415-1aea-4a11-9c5f-38283776c717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:17.837048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 907/1806] 10.222.0.93 () {58 vars in 1009 bytes} [Mon Aug 1 15:13:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:17.841798 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc39243-ff03-4f45-acf4-d8b22b6b437b None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:17.842590 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc39243-ff03-4f45-acf4-d8b22b6b437b None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:17.843256 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc39243-ff03-4f45-acf4-d8b22b6b437b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:17.852263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 900/1807] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:13:17 2022] DELETE /identity/v3/projects/cf289e1122ba465082fd2f9b7f5d388a => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:13:17.861304 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3e415-1aea-4a11-9c5f-38283776c717 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:17.868761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-4db3e415-1aea-4a11-9c5f-38283776c717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:17.904594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-2bc39243-ff03-4f45-acf4-d8b22b6b437b None None] MFA Rules not processed for user `69461e0bd4494c64b26331696ca0eb6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:17.907382 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-2bc39243-ff03-4f45-acf4-d8b22b6b437b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:18.096993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3e415-1aea-4a11-9c5f-38283776c717 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:18.122451 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626350d-065b-4c09-a7de-52015f160f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:18.128171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626350d-065b-4c09-a7de-52015f160f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:18.131516 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626350d-065b-4c09-a7de-52015f160f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:18.134343 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626350d-065b-4c09-a7de-52015f160f3b admin admin] PATH_INFO: `/v3/users/adb2d3a805d041c293585bf50f6f0ca3` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:18.140828 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4626350d-065b-4c09-a7de-52015f160f3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=adb2d3a805d041c293585bf50f6f0ca3)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:18.156446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4626350d-065b-4c09-a7de-52015f160f3b admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:18.230196 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 908/1808] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:18.231288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc39243-ff03-4f45-acf4-d8b22b6b437b None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:18.248750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-2bc39243-ff03-4f45-acf4-d8b22b6b437b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:18.349360 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc39243-ff03-4f45-acf4-d8b22b6b437b None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:18.355798 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4626350d-065b-4c09-a7de-52015f160f3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adb2d3a805d041c293585bf50f6f0ca3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:18.362003 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4626350d-065b-4c09-a7de-52015f160f3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adb2d3a805d041c293585bf50f6f0ca3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:18.368244 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f81512-df05-485c-b888-eaa309dba4c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:18.377302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f81512-df05-485c-b888-eaa309dba4c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:18.377753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f81512-df05-485c-b888-eaa309dba4c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:18.378040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f81512-df05-485c-b888-eaa309dba4c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:18.387375 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f0f81512-df05-485c-b888-eaa309dba4c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:18.395914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:18.419094 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4626350d-065b-4c09-a7de-52015f160f3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adb2d3a805d041c293585bf50f6f0ca3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:18.422670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4626350d-065b-4c09-a7de-52015f160f3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adb2d3a805d041c293585bf50f6f0ca3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:18.445193 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f81512-df05-485c-b888-eaa309dba4c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:18.462024 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f81512-df05-485c-b888-eaa309dba4c8 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:18.483239 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4626350d-065b-4c09-a7de-52015f160f3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adb2d3a805d041c293585bf50f6f0ca3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:18.528745 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 909/1809] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:18.533543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f81512-df05-485c-b888-eaa309dba4c8 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:18.538436 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f0f81512-df05-485c-b888-eaa309dba4c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:18.568592 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 901/1810] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:13:17 2022] DELETE /identity/v3/users/adb2d3a805d041c293585bf50f6f0ca3 => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:18.570248 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626350d-065b-4c09-a7de-52015f160f3b admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:18.580444 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-4626350d-065b-4c09-a7de-52015f160f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:18.654405 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f81512-df05-485c-b888-eaa309dba4c8 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:19.013259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626350d-065b-4c09-a7de-52015f160f3b admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:19.041169 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a272e-c892-4bf7-a360-9da19659f50a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:19.069783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a272e-c892-4bf7-a360-9da19659f50a service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:19.070278 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a272e-c892-4bf7-a360-9da19659f50a service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:19.072028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a272e-c892-4bf7-a360-9da19659f50a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:19.082193 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b412d-da14-4fca-8b98-fdcda61541ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:19.083367 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b412d-da14-4fca-8b98-fdcda61541ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:19.090674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-515a272e-c892-4bf7-a360-9da19659f50a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:19.095661 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b412d-da14-4fca-8b98-fdcda61541ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:19.096288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b412d-da14-4fca-8b98-fdcda61541ca admin admin] PATH_INFO: `/v3/users/9183cb952fa94bf6bbcf2771f11cd66e` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:19.106824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b412d-da14-4fca-8b98-fdcda61541ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=9183cb952fa94bf6bbcf2771f11cd66e)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:19.117186 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b412d-da14-4fca-8b98-fdcda61541ca admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:19.165852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:19.358268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-864b412d-da14-4fca-8b98-fdcda61541ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9183cb952fa94bf6bbcf2771f11cd66e'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:19.380280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-864b412d-da14-4fca-8b98-fdcda61541ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9183cb952fa94bf6bbcf2771f11cd66e'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:19.395008 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-864b412d-da14-4fca-8b98-fdcda61541ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9183cb952fa94bf6bbcf2771f11cd66e'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:19.396306 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-864b412d-da14-4fca-8b98-fdcda61541ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9183cb952fa94bf6bbcf2771f11cd66e'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:19.397516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-864b412d-da14-4fca-8b98-fdcda61541ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9183cb952fa94bf6bbcf2771f11cd66e'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:19.448029 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 910/1811] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:13:18 2022] DELETE /identity/v3/users/9183cb952fa94bf6bbcf2771f11cd66e => generated 0 bytes in 919 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:19.463219 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6eea2e-8627-496d-8fbc-5e02b3380e8b None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:19.469101 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6eea2e-8627-496d-8fbc-5e02b3380e8b None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:19.469661 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6eea2e-8627-496d-8fbc-5e02b3380e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:19.539117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a272e-c892-4bf7-a360-9da19659f50a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:19.546974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a272e-c892-4bf7-a360-9da19659f50a service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:19.548100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-ca6eea2e-8627-496d-8fbc-5e02b3380e8b None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:19.550714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ca6eea2e-8627-496d-8fbc-5e02b3380e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:19.611767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 902/1812] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 1042 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:19.853092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e341e22-8931-442b-86b0-01c891da8571 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:19.854168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e341e22-8931-442b-86b0-01c891da8571 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:19.854878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e341e22-8931-442b-86b0-01c891da8571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:19.899052 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 911/1813] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:19.902570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0a7d3-5ba1-4882-ab67-468aee32d0ea None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:19.903057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0a7d3-5ba1-4882-ab67-468aee32d0ea None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:19.903541 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0a7d3-5ba1-4882-ab67-468aee32d0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:19.993662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-58c0a7d3-5ba1-4882-ab67-468aee32d0ea None None] MFA Rules not processed for user `5a49850f6d8a474d83821f5f49dff06c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:19.996129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-58c0a7d3-5ba1-4882-ab67-468aee32d0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:20.111294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-3e341e22-8931-442b-86b0-01c891da8571 None None] MFA Rules not processed for user `4370f8f474c2446e989d0d8c5b75e87a`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:20.118403 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-3e341e22-8931-442b-86b0-01c891da8571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:20.334034 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 912/1814] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:20.335948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0a7d3-5ba1-4882-ab67-468aee32d0ea None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:20.340818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-58c0a7d3-5ba1-4882-ab67-468aee32d0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:20.446751 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:20.511317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:20.578700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 903/1815] 10.222.0.93 () {58 vars in 1009 bytes} [Mon Aug 1 15:13:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 727 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:13:20.583552 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0a7d3-5ba1-4882-ab67-468aee32d0ea None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:20.593767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c6e05-6e8c-4c3b-a379-d77caffc0694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:20.595076 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c6e05-6e8c-4c3b-a379-d77caffc0694 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:20.596633 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c6e05-6e8c-4c3b-a379-d77caffc0694 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:20.597332 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c6e05-6e8c-4c3b-a379-d77caffc0694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:20.625655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c6e05-6e8c-4c3b-a379-d77caffc0694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:20.630757 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c6e05-6e8c-4c3b-a379-d77caffc0694 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:20.735239 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 913/1816] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:21.036111 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e341e22-8931-442b-86b0-01c891da8571 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:21.042258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-3e341e22-8931-442b-86b0-01c891da8571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:21.534083 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e341e22-8931-442b-86b0-01c891da8571 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:21.560196 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241dcdc-6699-4e5a-9998-40429ab1c801 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4df37b02152c48509243b73ebbf49d8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:21.564713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241dcdc-6699-4e5a-9998-40429ab1c801 service swift] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:21.565801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241dcdc-6699-4e5a-9998-40429ab1c801 service swift] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:21.570688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241dcdc-6699-4e5a-9998-40429ab1c801 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:21.621775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241dcdc-6699-4e5a-9998-40429ab1c801 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:21.628772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241dcdc-6699-4e5a-9998-40429ab1c801 service swift] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:21.676487 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 904/1817] 10.222.0.93 () {60 vars in 1384 bytes} [Mon Aug 1 15:13:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 15:13:23.122392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049859b6-06d7-470c-8038-3cab23a91400 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:23.124261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049859b6-06d7-470c-8038-3cab23a91400 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:23.125086 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049859b6-06d7-470c-8038-3cab23a91400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:23.341661 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-049859b6-06d7-470c-8038-3cab23a91400 None None] MFA Rules not processed for user `4eba5f64833b482d81c53721c8442970`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:23.381853 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-049859b6-06d7-470c-8038-3cab23a91400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:23.492975 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 914/1818] 10.222.0.61 () {58 vars in 1015 bytes} [Mon Aug 1 15:13:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:23.627096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241dcdc-6699-4e5a-9998-40429ab1c801 service swift] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:23.637378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-9241dcdc-6699-4e5a-9998-40429ab1c801 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:23.745372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241dcdc-6699-4e5a-9998-40429ab1c801 service swift] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:23.812262 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b21b5c-e430-423a-9e1b-fb60887e9190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:23.820418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b21b5c-e430-423a-9e1b-fb60887e9190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:23.821028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b21b5c-e430-423a-9e1b-fb60887e9190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:23.821533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b21b5c-e430-423a-9e1b-fb60887e9190 admin admin] PATH_INFO: `/v3/projects/75198ce09a0747a9b77affbe1a66d891` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:23.832159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b21b5c-e430-423a-9e1b-fb60887e9190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75198ce09a0747a9b77affbe1a66d891)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:23.836875 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b21b5c-e430-423a-9e1b-fb60887e9190 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:24.044422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049859b6-06d7-470c-8038-3cab23a91400 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:24.046994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-049859b6-06d7-470c-8038-3cab23a91400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:24.106341 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049859b6-06d7-470c-8038-3cab23a91400 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:24.128570 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-73b21b5c-e430-423a-9e1b-fb60887e9190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75198ce09a0747a9b77affbe1a66d891', 'request_id': 'req-73b21b5c-e430-423a-9e1b-fb60887e9190'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:24.167906 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-73b21b5c-e430-423a-9e1b-fb60887e9190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75198ce09a0747a9b77affbe1a66d891', 'request_id': 'req-73b21b5c-e430-423a-9e1b-fb60887e9190'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:24.205240 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-73b21b5c-e430-423a-9e1b-fb60887e9190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75198ce09a0747a9b77affbe1a66d891', 'request_id': 'req-73b21b5c-e430-423a-9e1b-fb60887e9190'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:24.255801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-73b21b5c-e430-423a-9e1b-fb60887e9190 admin admin] The token cache is being invalidate because project 75198ce09a0747a9b77affbe1a66d891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:13:24.256911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-73b21b5c-e430-423a-9e1b-fb60887e9190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:24.258510 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-73b21b5c-e430-423a-9e1b-fb60887e9190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:24.263179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 905/1819] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:13:23 2022] DELETE /identity/v3/projects/75198ce09a0747a9b77affbe1a66d891 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:24.268993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b21b5c-e430-423a-9e1b-fb60887e9190 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:24.273518 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-73b21b5c-e430-423a-9e1b-fb60887e9190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:24.374210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcff08c-10cb-49e0-b5f8-1e225fa78203 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:24.375363 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcff08c-10cb-49e0-b5f8-1e225fa78203 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:24.376032 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcff08c-10cb-49e0-b5f8-1e225fa78203 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:24.376694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcff08c-10cb-49e0-b5f8-1e225fa78203 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:24.383365 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-7dcff08c-10cb-49e0-b5f8-1e225fa78203 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:24.439484 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcff08c-10cb-49e0-b5f8-1e225fa78203 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:24.442534 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcff08c-10cb-49e0-b5f8-1e225fa78203 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:24.494169 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 915/1820] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:24.496410 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcff08c-10cb-49e0-b5f8-1e225fa78203 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:24.513073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-7dcff08c-10cb-49e0-b5f8-1e225fa78203 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:24.531665 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:24.658442 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b21b5c-e430-423a-9e1b-fb60887e9190 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:24.693260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ced2ce-7e1c-4520-91e4-0c257189415b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:24.694563 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ced2ce-7e1c-4520-91e4-0c257189415b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:24.701454 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ced2ce-7e1c-4520-91e4-0c257189415b admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:24.703996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ced2ce-7e1c-4520-91e4-0c257189415b admin admin] PATH_INFO: `/v3/users/b9425868e7534a4e9c7d69a5ec7255a3` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:24.708934 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ced2ce-7e1c-4520-91e4-0c257189415b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9425868e7534a4e9c7d69a5ec7255a3)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:24.731151 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ced2ce-7e1c-4520-91e4-0c257189415b admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:24.909732 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-77ced2ce-7e1c-4520-91e4-0c257189415b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9425868e7534a4e9c7d69a5ec7255a3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:24.912075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-77ced2ce-7e1c-4520-91e4-0c257189415b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9425868e7534a4e9c7d69a5ec7255a3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:24.934301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcff08c-10cb-49e0-b5f8-1e225fa78203 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:24.947391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-77ced2ce-7e1c-4520-91e4-0c257189415b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9425868e7534a4e9c7d69a5ec7255a3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:24.950960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-77ced2ce-7e1c-4520-91e4-0c257189415b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9425868e7534a4e9c7d69a5ec7255a3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:24.966432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1e8b6-3f68-4642-a393-b86375f06756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:24.969875 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1e8b6-3f68-4642-a393-b86375f06756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:24.970930 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1e8b6-3f68-4642-a393-b86375f06756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:24.971698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1e8b6-3f68-4642-a393-b86375f06756 admin admin] PATH_INFO: `/v3/projects/edc96fdfcc5b46209755425c08260a52` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:24.982316 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1e8b6-3f68-4642-a393-b86375f06756 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edc96fdfcc5b46209755425c08260a52)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:24.993098 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1e8b6-3f68-4642-a393-b86375f06756 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:25.011494 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-77ced2ce-7e1c-4520-91e4-0c257189415b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9425868e7534a4e9c7d69a5ec7255a3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:25.131192 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 906/1821] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:13:24 2022] DELETE /identity/v3/users/b9425868e7534a4e9c7d69a5ec7255a3 => generated 0 bytes in 863 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:13:25.235793 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b8d1e8b6-3f68-4642-a393-b86375f06756 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edc96fdfcc5b46209755425c08260a52', 'request_id': 'req-b8d1e8b6-3f68-4642-a393-b86375f06756'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:25.261370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b8d1e8b6-3f68-4642-a393-b86375f06756 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edc96fdfcc5b46209755425c08260a52', 'request_id': 'req-b8d1e8b6-3f68-4642-a393-b86375f06756'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:25.291347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b8d1e8b6-3f68-4642-a393-b86375f06756 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edc96fdfcc5b46209755425c08260a52', 'request_id': 'req-b8d1e8b6-3f68-4642-a393-b86375f06756'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:25.331138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:25.344351 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b8d1e8b6-3f68-4642-a393-b86375f06756 admin admin] The token cache is being invalidate because project edc96fdfcc5b46209755425c08260a52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:13:25.345727 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b8d1e8b6-3f68-4642-a393-b86375f06756 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:25.349268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b8d1e8b6-3f68-4642-a393-b86375f06756 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:25.357092 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 916/1822] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:13:24 2022] DELETE /identity/v3/projects/edc96fdfcc5b46209755425c08260a52 => generated 0 bytes in 857 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:25.364911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48a403-bad3-4911-bbd7-481c309599fa None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:25.365508 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48a403-bad3-4911-bbd7-481c309599fa None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:25.366036 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48a403-bad3-4911-bbd7-481c309599fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:25.454320 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-1d48a403-bad3-4911-bbd7-481c309599fa None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:25.468420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-1d48a403-bad3-4911-bbd7-481c309599fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:25.655641 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 907/1823] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:25.666165 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b4149-9760-4703-a942-c6d54c374093 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:25.667075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b4149-9760-4703-a942-c6d54c374093 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:25.667075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b4149-9760-4703-a942-c6d54c374093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:25.708941 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-fb5b4149-9760-4703-a942-c6d54c374093 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:25.711762 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-fb5b4149-9760-4703-a942-c6d54c374093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:25.747442 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 917/1824] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:25.754177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48a403-bad3-4911-bbd7-481c309599fa None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:25.769223 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48a403-bad3-4911-bbd7-481c309599fa None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:25.778983 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6a9b6-27ce-4d28-8cc6-792d2e428f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:25.781374 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6a9b6-27ce-4d28-8cc6-792d2e428f3f admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:25.781928 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6a9b6-27ce-4d28-8cc6-792d2e428f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:25.782782 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6a9b6-27ce-4d28-8cc6-792d2e428f3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:25.784991 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6a9b6-27ce-4d28-8cc6-792d2e428f3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:25.785483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6a9b6-27ce-4d28-8cc6-792d2e428f3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:25.788243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6a9b6-27ce-4d28-8cc6-792d2e428f3f admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:25.820834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 908/1825] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:13:25.828860 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b4149-9760-4703-a942-c6d54c374093 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:25.858296 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b4149-9760-4703-a942-c6d54c374093 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:25.878457 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02579a-b582-476b-a5b6-d140c2261b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:25.880221 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02579a-b582-476b-a5b6-d140c2261b09 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:25.881155 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02579a-b582-476b-a5b6-d140c2261b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:25.881648 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02579a-b582-476b-a5b6-d140c2261b09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:25.882563 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02579a-b582-476b-a5b6-d140c2261b09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:25.887763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02579a-b582-476b-a5b6-d140c2261b09 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:25.982776 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 918/1826] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:13:25 2022] POST /identity/v3/projects => generated 375 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:13:25.990823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6a9b6-27ce-4d28-8cc6-792d2e428f3f admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:26.026863 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6a9b6-27ce-4d28-8cc6-792d2e428f3f admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:26.077146 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02579a-b582-476b-a5b6-d140c2261b09 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:26.082976 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-9e02579a-b582-476b-a5b6-d140c2261b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:26.158660 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:26.179878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8acd7f-869b-44db-a2bb-ea67652b5ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:26.180706 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8acd7f-869b-44db-a2bb-ea67652b5ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:26.181020 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8acd7f-869b-44db-a2bb-ea67652b5ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:26.182354 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8acd7f-869b-44db-a2bb-ea67652b5ca8 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:26.185006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8acd7f-869b-44db-a2bb-ea67652b5ca8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:26.192826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8acd7f-869b-44db-a2bb-ea67652b5ca8 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:26.263067 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 909/1827] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:13:25 2022] POST /identity/v3/users => generated 288 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:13:26.269599 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8acd7f-869b-44db-a2bb-ea67652b5ca8 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:26.304039 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8acd7f-869b-44db-a2bb-ea67652b5ca8 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:26.327788 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9fe47-68df-494f-a4dd-b513d5d87c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:26.328540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9fe47-68df-494f-a4dd-b513d5d87c40 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:26.328991 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9fe47-68df-494f-a4dd-b513d5d87c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:26.329470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9fe47-68df-494f-a4dd-b513d5d87c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:26.330604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9fe47-68df-494f-a4dd-b513d5d87c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:26.332133 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02579a-b582-476b-a5b6-d140c2261b09 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:26.333151 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9fe47-68df-494f-a4dd-b513d5d87c40 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:26.356301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b0902-cf65-45e3-89e7-e30f5f3279e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:26.357048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b0902-cf65-45e3-89e7-e30f5f3279e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:26.357341 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b0902-cf65-45e3-89e7-e30f5f3279e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:26.357597 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b0902-cf65-45e3-89e7-e30f5f3279e8 admin admin] PATH_INFO: `/v3/projects/23fb442a32bd4cf2999b1ca6a3f921ed` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:26.360448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b0902-cf65-45e3-89e7-e30f5f3279e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23fb442a32bd4cf2999b1ca6a3f921ed)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:26.364386 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b0902-cf65-45e3-89e7-e30f5f3279e8 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:26.412156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 910/1828] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:13:26.418353 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9fe47-68df-494f-a4dd-b513d5d87c40 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:26.464169 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9fe47-68df-494f-a4dd-b513d5d87c40 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:26.492296 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b6e81-ef15-4e3d-b08d-5f9e9a2ed4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:26.496078 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b6e81-ef15-4e3d-b08d-5f9e9a2ed4a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:26.498272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b6e81-ef15-4e3d-b08d-5f9e9a2ed4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:26.498679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b6e81-ef15-4e3d-b08d-5f9e9a2ed4a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:26.525123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b6e81-ef15-4e3d-b08d-5f9e9a2ed4a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:26.530978 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b6e81-ef15-4e3d-b08d-5f9e9a2ed4a9 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:26.597653 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 911/1829] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:13:26.598950 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b6e81-ef15-4e3d-b08d-5f9e9a2ed4a9 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:26.639742 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ae2b0902-cf65-45e3-89e7-e30f5f3279e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23fb442a32bd4cf2999b1ca6a3f921ed', 'request_id': 'req-ae2b0902-cf65-45e3-89e7-e30f5f3279e8'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:26.684332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b6e81-ef15-4e3d-b08d-5f9e9a2ed4a9 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:26.715517 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ae2b0902-cf65-45e3-89e7-e30f5f3279e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23fb442a32bd4cf2999b1ca6a3f921ed', 'request_id': 'req-ae2b0902-cf65-45e3-89e7-e30f5f3279e8'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:26.749590 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ae2b0902-cf65-45e3-89e7-e30f5f3279e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23fb442a32bd4cf2999b1ca6a3f921ed', 'request_id': 'req-ae2b0902-cf65-45e3-89e7-e30f5f3279e8'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:26.823488 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ae2b0902-cf65-45e3-89e7-e30f5f3279e8 admin admin] The token cache is being invalidate because project 23fb442a32bd4cf2999b1ca6a3f921ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:13:26.824649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ae2b0902-cf65-45e3-89e7-e30f5f3279e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:26.829941 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ae2b0902-cf65-45e3-89e7-e30f5f3279e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:26.841045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 919/1830] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:13:26 2022] DELETE /identity/v3/projects/23fb442a32bd4cf2999b1ca6a3f921ed => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:26.853339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926f049-eff9-407e-9e78-2ed8a5972b11 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:26.854450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926f049-eff9-407e-9e78-2ed8a5972b11 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:26.854978 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926f049-eff9-407e-9e78-2ed8a5972b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:27.050424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-9926f049-eff9-407e-9e78-2ed8a5972b11 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:27.066674 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-9926f049-eff9-407e-9e78-2ed8a5972b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:27.284074 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e8b97d-566b-4d69-b0f4-2b68902d115a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:27.294849 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8b97d-566b-4d69-b0f4-2b68902d115a admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:27.302482 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8b97d-566b-4d69-b0f4-2b68902d115a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:27.308682 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8b97d-566b-4d69-b0f4-2b68902d115a admin admin] PATH_INFO: `/v3/projects/b6a675927b28435295bdf40424a01ec5/users/18a22909a8a14106bbc8e5e09fab7bd5/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:27.492866 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e8b97d-566b-4d69-b0f4-2b68902d115a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6a675927b28435295bdf40424a01ec5, user_id=18a22909a8a14106bbc8e5e09fab7bd5, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:27.539328 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e8b97d-566b-4d69-b0f4-2b68902d115a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:27.750016 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 912/1831] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:13:26 2022] PUT /identity/v3/projects/b6a675927b28435295bdf40424a01ec5/users/18a22909a8a14106bbc8e5e09fab7bd5/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 1152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:13:27.793154 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d693b30-d223-4381-9655-efb557226806 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:27.800835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d693b30-d223-4381-9655-efb557226806 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:27.801350 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d693b30-d223-4381-9655-efb557226806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:27.936257 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 920/1832] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 1089 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:27.953636 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d08508-5801-452f-ae2e-928d4c4daa6f None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:27.954320 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d08508-5801-452f-ae2e-928d4c4daa6f None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:27.954796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d08508-5801-452f-ae2e-928d4c4daa6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:28.008995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:28.034189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-8d693b30-d223-4381-9655-efb557226806 None None] MFA Rules not processed for user `18a22909a8a14106bbc8e5e09fab7bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:28.049430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-8d693b30-d223-4381-9655-efb557226806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:28.063260 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-81d08508-5801-452f-ae2e-928d4c4daa6f None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:28.067625 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-81d08508-5801-452f-ae2e-928d4c4daa6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:28.275882 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 921/1833] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:28.282090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d08508-5801-452f-ae2e-928d4c4daa6f None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:28.398842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d08508-5801-452f-ae2e-928d4c4daa6f None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:28.473034 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af640d-75bb-430c-891d-e4af72857e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:28.488549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af640d-75bb-430c-891d-e4af72857e16 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:28.488549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af640d-75bb-430c-891d-e4af72857e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:28.488549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af640d-75bb-430c-891d-e4af72857e16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:28.501383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af640d-75bb-430c-891d-e4af72857e16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:28.507377 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af640d-75bb-430c-891d-e4af72857e16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:28.523585 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af640d-75bb-430c-891d-e4af72857e16 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:28.662418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 913/1834] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 888 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:28.686097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d693b30-d223-4381-9655-efb557226806 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:28.692388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-8d693b30-d223-4381-9655-efb557226806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:28.692798 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 922/1835] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:28.705168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af640d-75bb-430c-891d-e4af72857e16 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:28.808925 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af640d-75bb-430c-891d-e4af72857e16 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:28.881963 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372773fb-92ea-40ac-a26c-11ab328697f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:28.882680 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372773fb-92ea-40ac-a26c-11ab328697f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:28.883012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372773fb-92ea-40ac-a26c-11ab328697f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:28.883295 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372773fb-92ea-40ac-a26c-11ab328697f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:28.884368 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372773fb-92ea-40ac-a26c-11ab328697f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:28.915907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372773fb-92ea-40ac-a26c-11ab328697f6 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:29.082998 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 923/1836] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:13:28 2022] POST /identity/v3/projects => generated 373 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:13:29.092317 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfe5a3-a015-4cd5-91f6-abfaca63cb27 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:29.095434 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfe5a3-a015-4cd5-91f6-abfaca63cb27 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:29.096005 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfe5a3-a015-4cd5-91f6-abfaca63cb27 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:29.097781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 924/1837] 10.222.0.93 () {54 vars in 921 bytes} [Mon Aug 1 15:13:29 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:13:29.103053 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bfe5a3-a015-4cd5-91f6-abfaca63cb27 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:29.116276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:29.395373 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d693b30-d223-4381-9655-efb557226806 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:29.545711 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bfe5a3-a015-4cd5-91f6-abfaca63cb27 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:29.584630 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbbf5c-3adb-4870-8a2a-257744d38d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:29.588063 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbbf5c-3adb-4870-8a2a-257744d38d5c admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:29.591633 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbbf5c-3adb-4870-8a2a-257744d38d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:29.592222 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbbf5c-3adb-4870-8a2a-257744d38d5c admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:29.598251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbbf5c-3adb-4870-8a2a-257744d38d5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:29.617814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbbf5c-3adb-4870-8a2a-257744d38d5c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:29.737529 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27e2f1-6b28-4807-a76d-301857d339c5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4461a389da5848f5afd9854758707829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:29.740835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27e2f1-6b28-4807-a76d-301857d339c5 service glance] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:29.741456 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 925/1838] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:13:29 2022] POST /identity/v3/users => generated 287 bytes in 643 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:13:29.743753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27e2f1-6b28-4807-a76d-301857d339c5 service glance] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:29.746424 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27e2f1-6b28-4807-a76d-301857d339c5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:29.749148 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e815c0a-1b5c-4670-b3eb-2cf8f6c07668 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:29.749859 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e815c0a-1b5c-4670-b3eb-2cf8f6c07668 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:29.751175 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e815c0a-1b5c-4670-b3eb-2cf8f6c07668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:29.851932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-8e815c0a-1b5c-4670-b3eb-2cf8f6c07668 None None] MFA Rules not processed for user `3d9be74621d14c94b95ae94947e5abbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:29.864927 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-8e815c0a-1b5c-4670-b3eb-2cf8f6c07668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:30.064308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27e2f1-6b28-4807-a76d-301857d339c5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:30.070165 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27e2f1-6b28-4807-a76d-301857d339c5 service glance] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:30.098567 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 914/1839] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 1414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:30.099858 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27e2f1-6b28-4807-a76d-301857d339c5 service glance] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:30.122581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:30.124228 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27e2f1-6b28-4807-a76d-301857d339c5 service glance] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:30.153865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 926/1840] 10.222.0.93 () {58 vars in 1018 bytes} [Mon Aug 1 15:13:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:30.154259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c4911-2918-43a4-b0fc-cf82a0e97369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:30.156099 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c4911-2918-43a4-b0fc-cf82a0e97369 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:30.157085 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c4911-2918-43a4-b0fc-cf82a0e97369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:30.161117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c4911-2918-43a4-b0fc-cf82a0e97369 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:30.162603 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c4911-2918-43a4-b0fc-cf82a0e97369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:30.166065 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c4911-2918-43a4-b0fc-cf82a0e97369 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:30.176116 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e815c0a-1b5c-4670-b3eb-2cf8f6c07668 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:30.179286 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-8e815c0a-1b5c-4670-b3eb-2cf8f6c07668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:30.205842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e815c0a-1b5c-4670-b3eb-2cf8f6c07668 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:30.220479 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 915/1841] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:13:30.221622 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a702d-98d9-4e79-b5cb-1d6efde1973b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:30.222157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c4911-2918-43a4-b0fc-cf82a0e97369 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:30.222515 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a702d-98d9-4e79-b5cb-1d6efde1973b service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:30.222874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a702d-98d9-4e79-b5cb-1d6efde1973b service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:30.223360 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a702d-98d9-4e79-b5cb-1d6efde1973b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:30.247868 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a702d-98d9-4e79-b5cb-1d6efde1973b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:30.250340 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a702d-98d9-4e79-b5cb-1d6efde1973b service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:30.265095 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c4911-2918-43a4-b0fc-cf82a0e97369 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:30.281069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 927/1842] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:13:30.287843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae5fd0-5c76-47b1-b73b-6d2a2448a3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:30.288982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae5fd0-5c76-47b1-b73b-6d2a2448a3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:30.289482 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae5fd0-5c76-47b1-b73b-6d2a2448a3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:30.290205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae5fd0-5c76-47b1-b73b-6d2a2448a3e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:30.304845 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae5fd0-5c76-47b1-b73b-6d2a2448a3e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:30.309075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae5fd0-5c76-47b1-b73b-6d2a2448a3e5 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:30.349533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 916/1843] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:13:30.355777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a702d-98d9-4e79-b5cb-1d6efde1973b service nova] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:30.381382 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a702d-98d9-4e79-b5cb-1d6efde1973b service nova] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:30.395378 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2439d-fdba-444d-b607-2ea2cb842f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:30.406568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2439d-fdba-444d-b607-2ea2cb842f0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:30.407170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2439d-fdba-444d-b607-2ea2cb842f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:30.407730 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2439d-fdba-444d-b607-2ea2cb842f0d admin admin] PATH_INFO: `/v3/projects/d51ad2cf95f9437bbfc591999c7a4aa8/users/14611c117623417c99ab1a376f82681b/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:30.443123 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2439d-fdba-444d-b607-2ea2cb842f0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d51ad2cf95f9437bbfc591999c7a4aa8, user_id=14611c117623417c99ab1a376f82681b, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:30.452953 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2439d-fdba-444d-b607-2ea2cb842f0d admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:30.492539 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 928/1844] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:13:30 2022] PUT /identity/v3/projects/d51ad2cf95f9437bbfc591999c7a4aa8/users/14611c117623417c99ab1a376f82681b/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:30.493602 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5a405-c582-4ec2-8fee-dae32a7d211b None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:30.494147 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5a405-c582-4ec2-8fee-dae32a7d211b None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:30.496351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5a405-c582-4ec2-8fee-dae32a7d211b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:30.589151 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-2aa5a405-c582-4ec2-8fee-dae32a7d211b None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:30.590700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-2aa5a405-c582-4ec2-8fee-dae32a7d211b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:30.790813 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 917/1845] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:30.815783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2439d-fdba-444d-b607-2ea2cb842f0d admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:30.819253 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e1f2439d-fdba-444d-b607-2ea2cb842f0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:30.931645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2439d-fdba-444d-b607-2ea2cb842f0d admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:30.938242 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2416553-214c-4969-a5c6-8d1968ac0c5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:30.939046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2416553-214c-4969-a5c6-8d1968ac0c5a service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:30.939229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2416553-214c-4969-a5c6-8d1968ac0c5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:30.939568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2416553-214c-4969-a5c6-8d1968ac0c5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:30.955038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2416553-214c-4969-a5c6-8d1968ac0c5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:30.961250 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2416553-214c-4969-a5c6-8d1968ac0c5a service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:30.988360 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 929/1846] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:32.394092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a012f5-04a4-447d-beb1-2d83b59e7829 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:32.394631 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a012f5-04a4-447d-beb1-2d83b59e7829 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:32.395019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a012f5-04a4-447d-beb1-2d83b59e7829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:32.438186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:32.539290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-88a012f5-04a4-447d-beb1-2d83b59e7829 None None] MFA Rules not processed for user `14611c117623417c99ab1a376f82681b`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:32.543114 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-88a012f5-04a4-447d-beb1-2d83b59e7829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:32.814798 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 918/1847] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:32.817707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2416553-214c-4969-a5c6-8d1968ac0c5a service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:32.845012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2416553-214c-4969-a5c6-8d1968ac0c5a service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:32.862424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fa403-6358-4b07-a361-f4e0a9ff2c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:32.893334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fa403-6358-4b07-a361-f4e0a9ff2c2f admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:32.895137 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fa403-6358-4b07-a361-f4e0a9ff2c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:32.896713 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fa403-6358-4b07-a361-f4e0a9ff2c2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:32.899396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736fa403-6358-4b07-a361-f4e0a9ff2c2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:32.908184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736fa403-6358-4b07-a361-f4e0a9ff2c2f admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:33.002085 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 930/1848] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:13:32 2022] POST /identity/v3/projects => generated 373 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:13:33.010134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a012f5-04a4-447d-beb1-2d83b59e7829 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:33.046010 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a012f5-04a4-447d-beb1-2d83b59e7829 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:33.372723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19873950-dd19-47b7-a72b-39729d91e0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:33.374210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19873950-dd19-47b7-a72b-39729d91e0a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:33.374771 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19873950-dd19-47b7-a72b-39729d91e0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:33.375227 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19873950-dd19-47b7-a72b-39729d91e0a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:33.376151 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19873950-dd19-47b7-a72b-39729d91e0a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:33.381273 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19873950-dd19-47b7-a72b-39729d91e0a8 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:33.413878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 919/1849] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:13:33 2022] POST /identity/v3/users => generated 293 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:13:33.419931 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fa403-6358-4b07-a361-f4e0a9ff2c2f admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:33.457799 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fa403-6358-4b07-a361-f4e0a9ff2c2f admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:33.491555 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66e0fb-368f-4b40-86c1-4e32e1ff440b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:33.492634 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66e0fb-368f-4b40-86c1-4e32e1ff440b admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:33.493129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66e0fb-368f-4b40-86c1-4e32e1ff440b admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:33.493593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66e0fb-368f-4b40-86c1-4e32e1ff440b admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:33.504072 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66e0fb-368f-4b40-86c1-4e32e1ff440b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:33.505559 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66e0fb-368f-4b40-86c1-4e32e1ff440b admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:33.553949 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 931/1850] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:13:33.554208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19873950-dd19-47b7-a72b-39729d91e0a8 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:33.596388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19873950-dd19-47b7-a72b-39729d91e0a8 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:33.603550 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc108531-0c6a-4604-86a0-90456a03b98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:33.604984 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc108531-0c6a-4604-86a0-90456a03b98b admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:33.605492 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc108531-0c6a-4604-86a0-90456a03b98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:33.606134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc108531-0c6a-4604-86a0-90456a03b98b admin admin] PATH_INFO: `/v3/projects/0aa2bf6eaed047b99cf3b2758cda865a/users/8d8447d902a34ba8becb8ba5e208a715/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:33.640846 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc108531-0c6a-4604-86a0-90456a03b98b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aa2bf6eaed047b99cf3b2758cda865a, user_id=8d8447d902a34ba8becb8ba5e208a715, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:33.646787 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc108531-0c6a-4604-86a0-90456a03b98b admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:33.705845 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66e0fb-368f-4b40-86c1-4e32e1ff440b admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:33.706517 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 920/1851] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:13:33 2022] PUT /identity/v3/projects/0aa2bf6eaed047b99cf3b2758cda865a/users/8d8447d902a34ba8becb8ba5e208a715/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:13:33.737724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66e0fb-368f-4b40-86c1-4e32e1ff440b admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:33.827319 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a143c3-f5fc-4a65-81cf-9d8b6f57ea80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:33.827319 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a143c3-f5fc-4a65-81cf-9d8b6f57ea80 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:33.827319 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a143c3-f5fc-4a65-81cf-9d8b6f57ea80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:33.827319 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a143c3-f5fc-4a65-81cf-9d8b6f57ea80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:33.828886 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a143c3-f5fc-4a65-81cf-9d8b6f57ea80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:33.829645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a143c3-f5fc-4a65-81cf-9d8b6f57ea80 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:33.842115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 932/1852] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:13:33.848613 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc108531-0c6a-4604-86a0-90456a03b98b admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:33.860795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc108531-0c6a-4604-86a0-90456a03b98b admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:33.866334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536435f-a18d-4903-9cea-01698617e957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:33.867232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536435f-a18d-4903-9cea-01698617e957 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:33.867559 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536435f-a18d-4903-9cea-01698617e957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:33.867874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536435f-a18d-4903-9cea-01698617e957 admin admin] PATH_INFO: `/v3/projects/0aa2bf6eaed047b99cf3b2758cda865a/users/8d8447d902a34ba8becb8ba5e208a715/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:33.871115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0536435f-a18d-4903-9cea-01698617e957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aa2bf6eaed047b99cf3b2758cda865a, user_id=8d8447d902a34ba8becb8ba5e208a715, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:33.874096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0536435f-a18d-4903-9cea-01698617e957 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:33.888021 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 921/1853] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:13:33 2022] PUT /identity/v3/projects/0aa2bf6eaed047b99cf3b2758cda865a/users/8d8447d902a34ba8becb8ba5e208a715/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:34.554378 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29216a4-bdd5-4e44-b535-9178e68c719c None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:34.554847 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29216a4-bdd5-4e44-b535-9178e68c719c None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:34.555460 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29216a4-bdd5-4e44-b535-9178e68c719c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:34.606567 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-d29216a4-bdd5-4e44-b535-9178e68c719c None None] MFA Rules not processed for user `4370f8f474c2446e989d0d8c5b75e87a`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:34.609474 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-d29216a4-bdd5-4e44-b535-9178e68c719c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:34.636325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:34.713926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 933/1854] 10.222.0.93 () {58 vars in 1009 bytes} [Mon Aug 1 15:13:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:34.743348 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536435f-a18d-4903-9cea-01698617e957 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:34.747834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-0536435f-a18d-4903-9cea-01698617e957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:34.870750 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:34.938531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536435f-a18d-4903-9cea-01698617e957 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:34.949681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785310c-f331-4d56-b1eb-fdf2aa26e9e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4df37b02152c48509243b73ebbf49d8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:34.950834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785310c-f331-4d56-b1eb-fdf2aa26e9e1 service swift] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:34.951762 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785310c-f331-4d56-b1eb-fdf2aa26e9e1 service swift] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:34.952521 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785310c-f331-4d56-b1eb-fdf2aa26e9e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:34.971142 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785310c-f331-4d56-b1eb-fdf2aa26e9e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:34.977093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785310c-f331-4d56-b1eb-fdf2aa26e9e1 service swift] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:35.001760 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 922/1855] 10.222.0.93 () {60 vars in 1384 bytes} [Mon Aug 1 15:13:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:13:35.121052 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd585b0-b04e-47ff-b660-96cf72475e0a None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:35.122986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd585b0-b04e-47ff-b660-96cf72475e0a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:35.123807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd585b0-b04e-47ff-b660-96cf72475e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:35.189471 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-3bd585b0-b04e-47ff-b660-96cf72475e0a None None] MFA Rules not processed for user `8d8447d902a34ba8becb8ba5e208a715`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:35.192277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3bd585b0-b04e-47ff-b660-96cf72475e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:35.349656 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 934/1856] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:35 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:35.366586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785310c-f331-4d56-b1eb-fdf2aa26e9e1 service swift] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:35.368423 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e785310c-f331-4d56-b1eb-fdf2aa26e9e1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:35.469326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785310c-f331-4d56-b1eb-fdf2aa26e9e1 service swift] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:35.476690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3bcc8a-3ead-4cf2-9415-d5ee7bb61f7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:35.480697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bcc8a-3ead-4cf2-9415-d5ee7bb61f7f service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:35.480697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bcc8a-3ead-4cf2-9415-d5ee7bb61f7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:35.480697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bcc8a-3ead-4cf2-9415-d5ee7bb61f7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:35.609792 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3bcc8a-3ead-4cf2-9415-d5ee7bb61f7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:35.612816 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3bcc8a-3ead-4cf2-9415-d5ee7bb61f7f service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:35.632420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 923/1857] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:13:36.320810 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd585b0-b04e-47ff-b660-96cf72475e0a None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:36.323522 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3bd585b0-b04e-47ff-b660-96cf72475e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:36.758705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd585b0-b04e-47ff-b660-96cf72475e0a None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:36.794050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902dc6c-c1ff-45e1-96bd-cb2ed3189a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:36.799274 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902dc6c-c1ff-45e1-96bd-cb2ed3189a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:36.800355 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902dc6c-c1ff-45e1-96bd-cb2ed3189a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:36.807970 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902dc6c-c1ff-45e1-96bd-cb2ed3189a4e admin admin] PATH_INFO: `/v3/users/69461e0bd4494c64b26331696ca0eb6a` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:36.823748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902dc6c-c1ff-45e1-96bd-cb2ed3189a4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=69461e0bd4494c64b26331696ca0eb6a)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:36.829913 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902dc6c-c1ff-45e1-96bd-cb2ed3189a4e admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:37.099463 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-c902dc6c-c1ff-45e1-96bd-cb2ed3189a4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69461e0bd4494c64b26331696ca0eb6a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:37.112271 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:37.146090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-c902dc6c-c1ff-45e1-96bd-cb2ed3189a4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69461e0bd4494c64b26331696ca0eb6a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:37.196200 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-c902dc6c-c1ff-45e1-96bd-cb2ed3189a4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69461e0bd4494c64b26331696ca0eb6a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:37.202125 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-c902dc6c-c1ff-45e1-96bd-cb2ed3189a4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69461e0bd4494c64b26331696ca0eb6a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:37.212441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-c902dc6c-c1ff-45e1-96bd-cb2ed3189a4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69461e0bd4494c64b26331696ca0eb6a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:37.405411 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 935/1858] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:13:36 2022] DELETE /identity/v3/users/69461e0bd4494c64b26331696ca0eb6a => generated 0 bytes in 1082 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:37.408430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e68e67-e279-47b2-bc0d-cbd960051f77 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:37.409484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e68e67-e279-47b2-bc0d-cbd960051f77 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:37.410714 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e68e67-e279-47b2-bc0d-cbd960051f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:37.562712 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-25e68e67-e279-47b2-bc0d-cbd960051f77 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:37.565079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-25e68e67-e279-47b2-bc0d-cbd960051f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:37.789881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 924/1859] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:37.866173 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902dc6c-c1ff-45e1-96bd-cb2ed3189a4e admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:37.869684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-c902dc6c-c1ff-45e1-96bd-cb2ed3189a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:38.251180 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902dc6c-c1ff-45e1-96bd-cb2ed3189a4e admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:38.273062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce920c2c-395f-425c-8c64-98a6611c945c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:38.274046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce920c2c-395f-425c-8c64-98a6611c945c service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:38.274732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce920c2c-395f-425c-8c64-98a6611c945c service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:38.275795 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce920c2c-395f-425c-8c64-98a6611c945c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:38.312296 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce920c2c-395f-425c-8c64-98a6611c945c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:38.316401 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce920c2c-395f-425c-8c64-98a6611c945c service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:38.348025 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 936/1860] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:38.644904 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e68e67-e279-47b2-bc0d-cbd960051f77 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:38.647586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-25e68e67-e279-47b2-bc0d-cbd960051f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:38.913528 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:39.022339 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e68e67-e279-47b2-bc0d-cbd960051f77 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:39.032907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c8403-bfcb-47e0-b40d-2e092ecda3ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:39.034144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c8403-bfcb-47e0-b40d-2e092ecda3ac service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:39.034737 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c8403-bfcb-47e0-b40d-2e092ecda3ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:39.035302 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c8403-bfcb-47e0-b40d-2e092ecda3ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:39.042442 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-7d8c8403-bfcb-47e0-b40d-2e092ecda3ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:39.284156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c8403-bfcb-47e0-b40d-2e092ecda3ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:39.315810 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c8403-bfcb-47e0-b40d-2e092ecda3ac service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:39.394430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 925/1861] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 751 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:40.890482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9635a4-f3a8-4719-965d-7f72f968d594 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:40.891220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9635a4-f3a8-4719-965d-7f72f968d594 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:40.891394 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9635a4-f3a8-4719-965d-7f72f968d594 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:40.895108 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 937/1862] 10.222.0.93 () {54 vars in 921 bytes} [Mon Aug 1 15:13:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:13:41.146830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96133f1a-78c9-4b9d-b5f0-6faf45ec478c None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:41.147586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96133f1a-78c9-4b9d-b5f0-6faf45ec478c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:41.148187 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96133f1a-78c9-4b9d-b5f0-6faf45ec478c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:41.289861 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-96133f1a-78c9-4b9d-b5f0-6faf45ec478c None None] MFA Rules not processed for user `3d9be74621d14c94b95ae94947e5abbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:41.296314 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-96133f1a-78c9-4b9d-b5f0-6faf45ec478c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:41.536384 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 926/1863] 10.222.0.93 () {58 vars in 1018 bytes} [Mon Aug 1 15:13:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:41.627099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9635a4-f3a8-4719-965d-7f72f968d594 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:41.630171 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-8a9635a4-f3a8-4719-965d-7f72f968d594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:41.690632 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9635a4-f3a8-4719-965d-7f72f968d594 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:41.701637 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21388a55-c9de-4dc4-8dc8-a5d4d9e799e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:41.702352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21388a55-c9de-4dc4-8dc8-a5d4d9e799e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:41.702739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21388a55-c9de-4dc4-8dc8-a5d4d9e799e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:41.702884 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21388a55-c9de-4dc4-8dc8-a5d4d9e799e8 admin admin] PATH_INFO: `/v3/projects/f0871b19488d4b23b37dc5e1e776c3da` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:41.709268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21388a55-c9de-4dc4-8dc8-a5d4d9e799e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0871b19488d4b23b37dc5e1e776c3da)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:41.719356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21388a55-c9de-4dc4-8dc8-a5d4d9e799e8 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:41.924111 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-21388a55-c9de-4dc4-8dc8-a5d4d9e799e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0871b19488d4b23b37dc5e1e776c3da', 'request_id': 'req-21388a55-c9de-4dc4-8dc8-a5d4d9e799e8'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:41.965664 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-21388a55-c9de-4dc4-8dc8-a5d4d9e799e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0871b19488d4b23b37dc5e1e776c3da', 'request_id': 'req-21388a55-c9de-4dc4-8dc8-a5d4d9e799e8'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:41.979854 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-21388a55-c9de-4dc4-8dc8-a5d4d9e799e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0871b19488d4b23b37dc5e1e776c3da', 'request_id': 'req-21388a55-c9de-4dc4-8dc8-a5d4d9e799e8'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:42.038923 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-21388a55-c9de-4dc4-8dc8-a5d4d9e799e8 admin admin] The token cache is being invalidate because project f0871b19488d4b23b37dc5e1e776c3da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:13:42.039206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-21388a55-c9de-4dc4-8dc8-a5d4d9e799e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:42.042475 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-21388a55-c9de-4dc4-8dc8-a5d4d9e799e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:42.047401 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 938/1864] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:13:41 2022] DELETE /identity/v3/projects/f0871b19488d4b23b37dc5e1e776c3da => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:42.054887 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a5279-f393-4710-9c37-edd0f55ccb02 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:42.055837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a5279-f393-4710-9c37-edd0f55ccb02 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:42.057480 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a5279-f393-4710-9c37-edd0f55ccb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:42.101436 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-b91a5279-f393-4710-9c37-edd0f55ccb02 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:42.105492 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b91a5279-f393-4710-9c37-edd0f55ccb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:42.143776 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21388a55-c9de-4dc4-8dc8-a5d4d9e799e8 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:42.152700 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-21388a55-c9de-4dc4-8dc8-a5d4d9e799e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:42.303735 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:42.367123 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21388a55-c9de-4dc4-8dc8-a5d4d9e799e8 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:42.403229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad5cb6-393a-4e3f-b2a8-2c8c080cdac1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:42.408112 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad5cb6-393a-4e3f-b2a8-2c8c080cdac1 service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:42.410128 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad5cb6-393a-4e3f-b2a8-2c8c080cdac1 service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:42.410740 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad5cb6-393a-4e3f-b2a8-2c8c080cdac1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:42.416671 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-33ad5cb6-393a-4e3f-b2a8-2c8c080cdac1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:42.446269 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 927/1865] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:42.461444 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad64ad4-51af-4efa-9d60-9d28d7c47248 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:42.461953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad64ad4-51af-4efa-9d60-9d28d7c47248 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:42.462477 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad64ad4-51af-4efa-9d60-9d28d7c47248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:42.464690 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad5cb6-393a-4e3f-b2a8-2c8c080cdac1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:42.468554 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad5cb6-393a-4e3f-b2a8-2c8c080cdac1 service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:42.527552 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 939/1866] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:42.638779 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-1ad64ad4-51af-4efa-9d60-9d28d7c47248 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:42.640188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-1ad64ad4-51af-4efa-9d60-9d28d7c47248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:42.731100 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 928/1867] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:42.732797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad5cb6-393a-4e3f-b2a8-2c8c080cdac1 service nova] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:42.810741 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad5cb6-393a-4e3f-b2a8-2c8c080cdac1 service nova] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:42.846584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe54693-d01a-4f55-b95f-651bcdaf8a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:42.847695 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe54693-d01a-4f55-b95f-651bcdaf8a11 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:42.848200 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe54693-d01a-4f55-b95f-651bcdaf8a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:42.848799 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe54693-d01a-4f55-b95f-651bcdaf8a11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:42.856649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe54693-d01a-4f55-b95f-651bcdaf8a11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:42.857334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe54693-d01a-4f55-b95f-651bcdaf8a11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:42.867297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe54693-d01a-4f55-b95f-651bcdaf8a11 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:42.925885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 940/1868] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:42.931139 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad64ad4-51af-4efa-9d60-9d28d7c47248 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:42.949679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:42.971728 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad64ad4-51af-4efa-9d60-9d28d7c47248 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:42.992224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f671bf2-0364-4803-91e0-8a02946d670a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:42.993660 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f671bf2-0364-4803-91e0-8a02946d670a admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:42.994301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f671bf2-0364-4803-91e0-8a02946d670a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:42.997058 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f671bf2-0364-4803-91e0-8a02946d670a admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:43.000289 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f671bf2-0364-4803-91e0-8a02946d670a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:43.012719 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f671bf2-0364-4803-91e0-8a02946d670a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:43.093214 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 929/1869] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:13:42 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:13:43.098990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe54693-d01a-4f55-b95f-651bcdaf8a11 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:43.154211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe54693-d01a-4f55-b95f-651bcdaf8a11 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:43.487550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d7061-23d3-4fbd-8854-800f776e6235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:43.488530 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d7061-23d3-4fbd-8854-800f776e6235 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:43.488924 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d7061-23d3-4fbd-8854-800f776e6235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:43.497293 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d7061-23d3-4fbd-8854-800f776e6235 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:43.498505 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d7061-23d3-4fbd-8854-800f776e6235 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:43.501155 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d7061-23d3-4fbd-8854-800f776e6235 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:43.586885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 941/1870] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:13:43 2022] POST /identity/v3/users => generated 287 bytes in 488 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:13:43.592410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f671bf2-0364-4803-91e0-8a02946d670a admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:43.647807 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f671bf2-0364-4803-91e0-8a02946d670a admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:43.673341 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350b9e8-9f6e-44b9-999e-6cadae9c41a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:43.674900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350b9e8-9f6e-44b9-999e-6cadae9c41a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:43.675679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350b9e8-9f6e-44b9-999e-6cadae9c41a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:43.676411 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350b9e8-9f6e-44b9-999e-6cadae9c41a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:43.678233 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2350b9e8-9f6e-44b9-999e-6cadae9c41a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:43.683686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2350b9e8-9f6e-44b9-999e-6cadae9c41a7 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:43.729908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 930/1871] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:13:43.738009 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d7061-23d3-4fbd-8854-800f776e6235 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:43.784485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d7061-23d3-4fbd-8854-800f776e6235 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:43.797229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf2f36-9609-4a2f-9507-361577f930c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:43.801897 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf2f36-9609-4a2f-9507-361577f930c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:43.802507 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf2f36-9609-4a2f-9507-361577f930c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:43.803261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf2f36-9609-4a2f-9507-361577f930c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:43.806031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf2f36-9609-4a2f-9507-361577f930c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:43.813895 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf2f36-9609-4a2f-9507-361577f930c2 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:43.852854 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 942/1872] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:13:43.855934 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350b9e8-9f6e-44b9-999e-6cadae9c41a7 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:43.876860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350b9e8-9f6e-44b9-999e-6cadae9c41a7 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:43.895909 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652f306-15a2-4317-b6a2-9f383e9bd274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:43.897044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652f306-15a2-4317-b6a2-9f383e9bd274 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:43.897275 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652f306-15a2-4317-b6a2-9f383e9bd274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:43.897558 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652f306-15a2-4317-b6a2-9f383e9bd274 admin admin] PATH_INFO: `/v3/projects/ea5f354f1326410a8bca294ce81abf61/users/b7e7060f29af480faae12872f85bcede/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:43.963355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652f306-15a2-4317-b6a2-9f383e9bd274 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea5f354f1326410a8bca294ce81abf61, user_id=b7e7060f29af480faae12872f85bcede, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:43.966264 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652f306-15a2-4317-b6a2-9f383e9bd274 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:44.002175 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 931/1873] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:13:43 2022] PUT /identity/v3/projects/ea5f354f1326410a8bca294ce81abf61/users/b7e7060f29af480faae12872f85bcede/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:13:44.014999 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f54dfa-88d5-4493-8cf9-da60f1f9e6d3 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:44.016770 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f54dfa-88d5-4493-8cf9-da60f1f9e6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:44.018287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f54dfa-88d5-4493-8cf9-da60f1f9e6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:44.150870 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-b7f54dfa-88d5-4493-8cf9-da60f1f9e6d3 None None] MFA Rules not processed for user `b7e7060f29af480faae12872f85bcede`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:44.153170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b7f54dfa-88d5-4493-8cf9-da60f1f9e6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:44.578161 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 943/1874] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 564 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:44.600256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652f306-15a2-4317-b6a2-9f383e9bd274 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:44.603073 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-1652f306-15a2-4317-b6a2-9f383e9bd274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:44.885222 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652f306-15a2-4317-b6a2-9f383e9bd274 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:44.950074 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2c947-dda5-47f0-8ad2-7bbca17d4dcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:44.955308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2c947-dda5-47f0-8ad2-7bbca17d4dcf service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:44.955987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2c947-dda5-47f0-8ad2-7bbca17d4dcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:44.956502 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2c947-dda5-47f0-8ad2-7bbca17d4dcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:44.999114 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2c947-dda5-47f0-8ad2-7bbca17d4dcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:45.003597 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2c947-dda5-47f0-8ad2-7bbca17d4dcf service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:45.043197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 932/1875] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:13:51.011633 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f54dfa-88d5-4493-8cf9-da60f1f9e6d3 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:51.080817 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:51.140611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f54dfa-88d5-4493-8cf9-da60f1f9e6d3 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:51.146008 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09707a9-4cb5-4260-9750-fb8a56f0049c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:51.146730 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09707a9-4cb5-4260-9750-fb8a56f0049c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:51.147030 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09707a9-4cb5-4260-9750-fb8a56f0049c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:51.147285 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09707a9-4cb5-4260-9750-fb8a56f0049c admin admin] PATH_INFO: `/v3/users/b7e7060f29af480faae12872f85bcede` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:51.153510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09707a9-4cb5-4260-9750-fb8a56f0049c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7e7060f29af480faae12872f85bcede)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:51.155978 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09707a9-4cb5-4260-9750-fb8a56f0049c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:51.241223 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e09707a9-4cb5-4260-9750-fb8a56f0049c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7e7060f29af480faae12872f85bcede'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:51.252524 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e09707a9-4cb5-4260-9750-fb8a56f0049c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7e7060f29af480faae12872f85bcede'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:51.264424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e09707a9-4cb5-4260-9750-fb8a56f0049c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7e7060f29af480faae12872f85bcede'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:51.266453 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e09707a9-4cb5-4260-9750-fb8a56f0049c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7e7060f29af480faae12872f85bcede'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:51.267614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e09707a9-4cb5-4260-9750-fb8a56f0049c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7e7060f29af480faae12872f85bcede'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:51.302280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 944/1876] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:13:51 2022] DELETE /identity/v3/users/b7e7060f29af480faae12872f85bcede => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:51.308954 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d92e5-dfab-4550-940a-09f341e414cf None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:51.309899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d92e5-dfab-4550-940a-09f341e414cf None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:51.309899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d92e5-dfab-4550-940a-09f341e414cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:51.334200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-ef2d92e5-dfab-4550-940a-09f341e414cf None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:51.337140 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ef2d92e5-dfab-4550-940a-09f341e414cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:51.469883 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 933/1877] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:51.484318 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09707a9-4cb5-4260-9750-fb8a56f0049c admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:51.486574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e09707a9-4cb5-4260-9750-fb8a56f0049c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:51.641392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09707a9-4cb5-4260-9750-fb8a56f0049c admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:51.654548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db6d0a8-0e5a-4db7-b854-1983241d4e4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:51.655695 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6d0a8-0e5a-4db7-b854-1983241d4e4a service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:51.656191 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6d0a8-0e5a-4db7-b854-1983241d4e4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:51.656710 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6d0a8-0e5a-4db7-b854-1983241d4e4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:51.686195 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db6d0a8-0e5a-4db7-b854-1983241d4e4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:51.690187 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db6d0a8-0e5a-4db7-b854-1983241d4e4a service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:51.706020 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 945/1878] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:13:52.331666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d92e5-dfab-4550-940a-09f341e414cf None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:52.336262 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ef2d92e5-dfab-4550-940a-09f341e414cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:52.359956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d92e5-dfab-4550-940a-09f341e414cf None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:52.368429 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef75d9-547a-455f-9e7b-5608c7eeb591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:52.370465 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef75d9-547a-455f-9e7b-5608c7eeb591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:52.371197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef75d9-547a-455f-9e7b-5608c7eeb591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:52.372019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef75d9-547a-455f-9e7b-5608c7eeb591 admin admin] PATH_INFO: `/v3/projects/ea5f354f1326410a8bca294ce81abf61` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:52.374506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef75d9-547a-455f-9e7b-5608c7eeb591 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea5f354f1326410a8bca294ce81abf61)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:52.382749 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef75d9-547a-455f-9e7b-5608c7eeb591 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:52.453625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:52.464728 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c0ef75d9-547a-455f-9e7b-5608c7eeb591 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea5f354f1326410a8bca294ce81abf61', 'request_id': 'req-c0ef75d9-547a-455f-9e7b-5608c7eeb591'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:52.480409 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c0ef75d9-547a-455f-9e7b-5608c7eeb591 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea5f354f1326410a8bca294ce81abf61', 'request_id': 'req-c0ef75d9-547a-455f-9e7b-5608c7eeb591'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:52.500755 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c0ef75d9-547a-455f-9e7b-5608c7eeb591 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea5f354f1326410a8bca294ce81abf61', 'request_id': 'req-c0ef75d9-547a-455f-9e7b-5608c7eeb591'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:52.532384 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c0ef75d9-547a-455f-9e7b-5608c7eeb591 admin admin] The token cache is being invalidate because project ea5f354f1326410a8bca294ce81abf61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:13:52.532941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c0ef75d9-547a-455f-9e7b-5608c7eeb591 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:52.534730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c0ef75d9-547a-455f-9e7b-5608c7eeb591 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:52.542061 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 934/1879] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:13:52 2022] DELETE /identity/v3/projects/ea5f354f1326410a8bca294ce81abf61 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:52.543866 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179752aa-f6a0-44a3-9901-3913026ed776 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:52.544338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179752aa-f6a0-44a3-9901-3913026ed776 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:52.544427 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179752aa-f6a0-44a3-9901-3913026ed776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:52.576040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-179752aa-f6a0-44a3-9901-3913026ed776 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:52.577630 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-179752aa-f6a0-44a3-9901-3913026ed776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:52.652739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:52.684242 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 946/1880] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:52.689671 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903a1b7-a666-4234-a7d9-86ffdc134d54 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:52.690025 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903a1b7-a666-4234-a7d9-86ffdc134d54 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:52.690377 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903a1b7-a666-4234-a7d9-86ffdc134d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:52.716890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-b903a1b7-a666-4234-a7d9-86ffdc134d54 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:52.717325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b903a1b7-a666-4234-a7d9-86ffdc134d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:52.730540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 935/1881] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:52.731088 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179752aa-f6a0-44a3-9901-3913026ed776 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:52.750874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179752aa-f6a0-44a3-9901-3913026ed776 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:52.757433 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9bdc3-cfad-4f58-b28a-37093125bcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:52.758930 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9bdc3-cfad-4f58-b28a-37093125bcf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:52.760838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9bdc3-cfad-4f58-b28a-37093125bcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:52.761188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9bdc3-cfad-4f58-b28a-37093125bcf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:52.762055 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9bdc3-cfad-4f58-b28a-37093125bcf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:52.762482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9bdc3-cfad-4f58-b28a-37093125bcf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:52.765225 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9bdc3-cfad-4f58-b28a-37093125bcf4 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:52.801732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 947/1882] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:13:52.816277 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903a1b7-a666-4234-a7d9-86ffdc134d54 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:52.836825 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903a1b7-a666-4234-a7d9-86ffdc134d54 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:52.849762 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da5cbd-378c-4b7a-bd8b-0acb23d3cf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:52.849762 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da5cbd-378c-4b7a-bd8b-0acb23d3cf82 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:52.850050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da5cbd-378c-4b7a-bd8b-0acb23d3cf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:52.850215 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da5cbd-378c-4b7a-bd8b-0acb23d3cf82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:52.851093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da5cbd-378c-4b7a-bd8b-0acb23d3cf82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:52.853219 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da5cbd-378c-4b7a-bd8b-0acb23d3cf82 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:52.907734 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 936/1883] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:13:52 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:13:52.910478 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9bdc3-cfad-4f58-b28a-37093125bcf4 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:52.935388 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9bdc3-cfad-4f58-b28a-37093125bcf4 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:53.128958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5d593-0608-429c-9d69-b0e2b1887099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:53.130026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5d593-0608-429c-9d69-b0e2b1887099 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:53.130569 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5d593-0608-429c-9d69-b0e2b1887099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:53.131062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5d593-0608-429c-9d69-b0e2b1887099 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:53.132329 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5d593-0608-429c-9d69-b0e2b1887099 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:53.135548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5d593-0608-429c-9d69-b0e2b1887099 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:53.214231 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 948/1884] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:13:52 2022] POST /identity/v3/users => generated 293 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:13:53.220693 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da5cbd-378c-4b7a-bd8b-0acb23d3cf82 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:53.281384 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da5cbd-378c-4b7a-bd8b-0acb23d3cf82 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:53.316687 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb9590-f555-464d-8dd4-cf1573630e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:53.320126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb9590-f555-464d-8dd4-cf1573630e91 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:53.322666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb9590-f555-464d-8dd4-cf1573630e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:53.324768 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb9590-f555-464d-8dd4-cf1573630e91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:53.326921 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb9590-f555-464d-8dd4-cf1573630e91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:53.348559 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb9590-f555-464d-8dd4-cf1573630e91 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:53.396997 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 937/1885] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:13:53.402394 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5d593-0608-429c-9d69-b0e2b1887099 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:53.426046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5d593-0608-429c-9d69-b0e2b1887099 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:53.439657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6517a-6195-4b7c-9fde-dde917fbe928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:53.440326 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6517a-6195-4b7c-9fde-dde917fbe928 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:53.440634 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6517a-6195-4b7c-9fde-dde917fbe928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:53.440938 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6517a-6195-4b7c-9fde-dde917fbe928 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:53.445004 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6517a-6195-4b7c-9fde-dde917fbe928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:53.449088 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6517a-6195-4b7c-9fde-dde917fbe928 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:53.507017 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb9590-f555-464d-8dd4-cf1573630e91 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:53.508443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 949/1886] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:13:53.526551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb9590-f555-464d-8dd4-cf1573630e91 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:53.552703 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda8b1e-f006-4019-b30e-3077e227769c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:53.554119 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda8b1e-f006-4019-b30e-3077e227769c admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:53.555042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda8b1e-f006-4019-b30e-3077e227769c admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:53.555566 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda8b1e-f006-4019-b30e-3077e227769c admin admin] PATH_INFO: `/v3/projects/b03c6bfc69444050a915366fb4e2dfc2/users/c366195f967f40ac962dde911adb5f35/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:53.589712 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda8b1e-f006-4019-b30e-3077e227769c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b03c6bfc69444050a915366fb4e2dfc2, user_id=c366195f967f40ac962dde911adb5f35, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:53.597824 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda8b1e-f006-4019-b30e-3077e227769c admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:53.633878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 938/1887] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:13:53 2022] PUT /identity/v3/projects/b03c6bfc69444050a915366fb4e2dfc2/users/c366195f967f40ac962dde911adb5f35/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:53.646531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ff725-b0aa-4811-82bf-ecd1f5c96647 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:53.647402 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ff725-b0aa-4811-82bf-ecd1f5c96647 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:53.647402 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ff725-b0aa-4811-82bf-ecd1f5c96647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:53.721730 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-0d1ff725-b0aa-4811-82bf-ecd1f5c96647 None None] MFA Rules not processed for user `c366195f967f40ac962dde911adb5f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:53.723870 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0d1ff725-b0aa-4811-82bf-ecd1f5c96647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:53.881465 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 950/1888] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:13:53.893114 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda8b1e-f006-4019-b30e-3077e227769c admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:53.896006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-beda8b1e-f006-4019-b30e-3077e227769c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:54.125443 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda8b1e-f006-4019-b30e-3077e227769c admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:54.143021 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dab825-9cec-4c0e-801f-d4e1880199b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:54.144357 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dab825-9cec-4c0e-801f-d4e1880199b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:54.145052 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dab825-9cec-4c0e-801f-d4e1880199b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:54.145618 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dab825-9cec-4c0e-801f-d4e1880199b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:54.170649 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dab825-9cec-4c0e-801f-d4e1880199b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:54.176746 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dab825-9cec-4c0e-801f-d4e1880199b7 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:54.228976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 939/1889] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:55.427351 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ff725-b0aa-4811-82bf-ecd1f5c96647 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:55.433997 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0d1ff725-b0aa-4811-82bf-ecd1f5c96647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:55.667416 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:55.713701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ff725-b0aa-4811-82bf-ecd1f5c96647 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:55.724688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff312de-bab9-44ee-a52e-f8baf8472a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:55.724927 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff312de-bab9-44ee-a52e-f8baf8472a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:55.725421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff312de-bab9-44ee-a52e-f8baf8472a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:55.725964 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff312de-bab9-44ee-a52e-f8baf8472a98 admin admin] PATH_INFO: `/v3/users/915d10d6f93841c291b27056e02d2db8` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:55.729251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff312de-bab9-44ee-a52e-f8baf8472a98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=915d10d6f93841c291b27056e02d2db8)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:55.733974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff312de-bab9-44ee-a52e-f8baf8472a98 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:55.838144 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6ff312de-bab9-44ee-a52e-f8baf8472a98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '915d10d6f93841c291b27056e02d2db8'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:55.907640 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6ff312de-bab9-44ee-a52e-f8baf8472a98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '915d10d6f93841c291b27056e02d2db8'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:55.928957 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6ff312de-bab9-44ee-a52e-f8baf8472a98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '915d10d6f93841c291b27056e02d2db8'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:55.934171 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6ff312de-bab9-44ee-a52e-f8baf8472a98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '915d10d6f93841c291b27056e02d2db8'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:55.938600 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6ff312de-bab9-44ee-a52e-f8baf8472a98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '915d10d6f93841c291b27056e02d2db8'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:56.008567 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 951/1890] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:13:55 2022] DELETE /identity/v3/users/915d10d6f93841c291b27056e02d2db8 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:56.016244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865f3aa-cb62-451d-91bc-bcc586bc463a None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:56.018280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865f3aa-cb62-451d-91bc-bcc586bc463a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:56.018280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865f3aa-cb62-451d-91bc-bcc586bc463a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:56.059510 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-5865f3aa-cb62-451d-91bc-bcc586bc463a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:56.062073 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-5865f3aa-cb62-451d-91bc-bcc586bc463a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:56.359855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 940/1891] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:13:56.390543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff312de-bab9-44ee-a52e-f8baf8472a98 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:56.392712 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-6ff312de-bab9-44ee-a52e-f8baf8472a98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:56.647375 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff312de-bab9-44ee-a52e-f8baf8472a98 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:56.657372 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8e8d2-7040-4f74-b999-d3e3d1dc6d6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:56.658547 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8e8d2-7040-4f74-b999-d3e3d1dc6d6e service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:56.659017 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8e8d2-7040-4f74-b999-d3e3d1dc6d6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:56.659432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8e8d2-7040-4f74-b999-d3e3d1dc6d6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:56.699553 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8e8d2-7040-4f74-b999-d3e3d1dc6d6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:56.706961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8e8d2-7040-4f74-b999-d3e3d1dc6d6e service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:56.737814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 952/1892] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:57.215703 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865f3aa-cb62-451d-91bc-bcc586bc463a None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:57.219478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-5865f3aa-cb62-451d-91bc-bcc586bc463a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:57.256825 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865f3aa-cb62-451d-91bc-bcc586bc463a None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:57.281821 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d97784-c2d0-44c4-96ef-d90f91f11c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:57.284572 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d97784-c2d0-44c4-96ef-d90f91f11c5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:57.286793 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d97784-c2d0-44c4-96ef-d90f91f11c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:57.289146 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d97784-c2d0-44c4-96ef-d90f91f11c5d admin admin] PATH_INFO: `/v3/projects/48377ffae6d3402ca336a83bb8024db1` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:57.295334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d97784-c2d0-44c4-96ef-d90f91f11c5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=48377ffae6d3402ca336a83bb8024db1)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:57.304721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d97784-c2d0-44c4-96ef-d90f91f11c5d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:57.345638 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:57.480582 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-b6d97784-c2d0-44c4-96ef-d90f91f11c5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48377ffae6d3402ca336a83bb8024db1', 'request_id': 'req-b6d97784-c2d0-44c4-96ef-d90f91f11c5d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:57.482029 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-b6d97784-c2d0-44c4-96ef-d90f91f11c5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48377ffae6d3402ca336a83bb8024db1', 'request_id': 'req-b6d97784-c2d0-44c4-96ef-d90f91f11c5d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:57.500530 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-b6d97784-c2d0-44c4-96ef-d90f91f11c5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48377ffae6d3402ca336a83bb8024db1', 'request_id': 'req-b6d97784-c2d0-44c4-96ef-d90f91f11c5d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:57.569072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-b6d97784-c2d0-44c4-96ef-d90f91f11c5d admin admin] The token cache is being invalidate because project 48377ffae6d3402ca336a83bb8024db1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:13:57.569580 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-b6d97784-c2d0-44c4-96ef-d90f91f11c5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:57.575175 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-b6d97784-c2d0-44c4-96ef-d90f91f11c5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:57.583786 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 941/1893] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:13:57 2022] DELETE /identity/v3/projects/48377ffae6d3402ca336a83bb8024db1 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:13:57.590204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8e8d2-7040-4f74-b999-d3e3d1dc6d6e service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:57.622842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8e8d2-7040-4f74-b999-d3e3d1dc6d6e service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:57.786344 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d97784-c2d0-44c4-96ef-d90f91f11c5d admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:57.788962 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b6d97784-c2d0-44c4-96ef-d90f91f11c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:57.884973 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea8544-5249-45ab-bcc0-3aec5945422c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:57.885661 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea8544-5249-45ab-bcc0-3aec5945422c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:57.885936 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea8544-5249-45ab-bcc0-3aec5945422c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:57.887162 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea8544-5249-45ab-bcc0-3aec5945422c admin admin] PATH_INFO: `/v3/users/7512357fd3da4e6cb52a490c267fd004` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:57.889489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea8544-5249-45ab-bcc0-3aec5945422c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7512357fd3da4e6cb52a490c267fd004)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:57.891864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea8544-5249-45ab-bcc0-3aec5945422c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:57.929217 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:57.992051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d97784-c2d0-44c4-96ef-d90f91f11c5d admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:58.007310 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff819c7-f869-4457-8d44-d74a92010ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:58.008370 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff819c7-f869-4457-8d44-d74a92010ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:58.010325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff819c7-f869-4457-8d44-d74a92010ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:58.012040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff819c7-f869-4457-8d44-d74a92010ced admin admin] PATH_INFO: `/v3/users/c366195f967f40ac962dde911adb5f35` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:58.014720 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fbea8544-5249-45ab-bcc0-3aec5945422c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7512357fd3da4e6cb52a490c267fd004'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:58.016903 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff819c7-f869-4457-8d44-d74a92010ced admin admin] RBAC: Authorizing `identity:delete_user(user_id=c366195f967f40ac962dde911adb5f35)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:58.022755 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff819c7-f869-4457-8d44-d74a92010ced admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:58.034993 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fbea8544-5249-45ab-bcc0-3aec5945422c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7512357fd3da4e6cb52a490c267fd004'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:58.056579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fbea8544-5249-45ab-bcc0-3aec5945422c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7512357fd3da4e6cb52a490c267fd004'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:58.057749 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fbea8544-5249-45ab-bcc0-3aec5945422c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7512357fd3da4e6cb52a490c267fd004'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:58.059547 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fbea8544-5249-45ab-bcc0-3aec5945422c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7512357fd3da4e6cb52a490c267fd004'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:58.122001 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 953/1894] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:13:57 2022] DELETE /identity/v3/users/7512357fd3da4e6cb52a490c267fd004 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:58.125802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95574b7a-de45-4f44-84e2-002849e4e330 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:58.127830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95574b7a-de45-4f44-84e2-002849e4e330 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:58.128175 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95574b7a-de45-4f44-84e2-002849e4e330 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:58.133325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 954/1895] 10.222.0.93 () {54 vars in 921 bytes} [Mon Aug 1 15:13:58 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:13:58.145562 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-8ff819c7-f869-4457-8d44-d74a92010ced admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c366195f967f40ac962dde911adb5f35'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:58.147170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-8ff819c7-f869-4457-8d44-d74a92010ced admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c366195f967f40ac962dde911adb5f35'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:58.183611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-8ff819c7-f869-4457-8d44-d74a92010ced admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c366195f967f40ac962dde911adb5f35'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:58.185534 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-8ff819c7-f869-4457-8d44-d74a92010ced admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c366195f967f40ac962dde911adb5f35'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:58.230443 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-8ff819c7-f869-4457-8d44-d74a92010ced admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c366195f967f40ac962dde911adb5f35'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:58.263986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5d712-c637-4b0a-8a92-03558ec1da4b None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:58.263986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5d712-c637-4b0a-8a92-03558ec1da4b None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:58.264321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5d712-c637-4b0a-8a92-03558ec1da4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:58.267851 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 942/1896] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:13:57 2022] DELETE /identity/v3/users/c366195f967f40ac962dde911adb5f35 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:58.323592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-93c5d712-c637-4b0a-8a92-03558ec1da4b None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:58.329170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-93c5d712-c637-4b0a-8a92-03558ec1da4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:58.338763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc4b99-6e26-41bc-ac18-42a60e345acd None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:58.339148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc4b99-6e26-41bc-ac18-42a60e345acd None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:58.339450 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc4b99-6e26-41bc-ac18-42a60e345acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:58.401302 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-5dbc4b99-6e26-41bc-ac18-42a60e345acd None None] MFA Rules not processed for user `3d9be74621d14c94b95ae94947e5abbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:58.403103 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-5dbc4b99-6e26-41bc-ac18-42a60e345acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:58.442120 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:58.518962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 955/1897] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:58.554923 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5d712-c637-4b0a-8a92-03558ec1da4b None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:58.560025 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-93c5d712-c637-4b0a-8a92-03558ec1da4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:58.602089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 943/1898] 10.222.0.93 () {58 vars in 1018 bytes} [Mon Aug 1 15:13:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:13:58.838112 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5d712-c637-4b0a-8a92-03558ec1da4b None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:58.854116 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f60b0-fa17-4b1c-906e-98bcecc8ad85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:58.856246 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f60b0-fa17-4b1c-906e-98bcecc8ad85 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:58.856850 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f60b0-fa17-4b1c-906e-98bcecc8ad85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:58.859324 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f60b0-fa17-4b1c-906e-98bcecc8ad85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:58.925002 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f60b0-fa17-4b1c-906e-98bcecc8ad85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:58.929103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f60b0-fa17-4b1c-906e-98bcecc8ad85 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:58.964332 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 956/1899] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:59.283858 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc4b99-6e26-41bc-ac18-42a60e345acd None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:59.287668 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-5dbc4b99-6e26-41bc-ac18-42a60e345acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:59.346945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc4b99-6e26-41bc-ac18-42a60e345acd None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:59.356993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1b486-d61f-4198-a3ca-38c2181c5121 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:59.357652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1b486-d61f-4198-a3ca-38c2181c5121 service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:59.357936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1b486-d61f-4198-a3ca-38c2181c5121 service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:59.358341 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1b486-d61f-4198-a3ca-38c2181c5121 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:59.373728 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1b486-d61f-4198-a3ca-38c2181c5121 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:59.376044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1b486-d61f-4198-a3ca-38c2181c5121 service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:59.397889 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 944/1900] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:13:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:13:59.419435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f60b0-fa17-4b1c-906e-98bcecc8ad85 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:59.422100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-d15f60b0-fa17-4b1c-906e-98bcecc8ad85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:59.452147 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f60b0-fa17-4b1c-906e-98bcecc8ad85 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:59.470973 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b4d8e-38bf-4dd4-b119-68592a3cfe90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:59.472190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b4d8e-38bf-4dd4-b119-68592a3cfe90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:59.472899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b4d8e-38bf-4dd4-b119-68592a3cfe90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:59.473433 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b4d8e-38bf-4dd4-b119-68592a3cfe90 admin admin] PATH_INFO: `/v3/projects/4a1093da0f6d4ce69499a561d173fcbd` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:59.475701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b4d8e-38bf-4dd4-b119-68592a3cfe90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a1093da0f6d4ce69499a561d173fcbd)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:59.478300 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b4d8e-38bf-4dd4-b119-68592a3cfe90 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:59.544879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:59.593491 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a97b4d8e-38bf-4dd4-b119-68592a3cfe90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a1093da0f6d4ce69499a561d173fcbd', 'request_id': 'req-a97b4d8e-38bf-4dd4-b119-68592a3cfe90'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:59.611205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a97b4d8e-38bf-4dd4-b119-68592a3cfe90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a1093da0f6d4ce69499a561d173fcbd', 'request_id': 'req-a97b4d8e-38bf-4dd4-b119-68592a3cfe90'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:59.622596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a97b4d8e-38bf-4dd4-b119-68592a3cfe90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a1093da0f6d4ce69499a561d173fcbd', 'request_id': 'req-a97b4d8e-38bf-4dd4-b119-68592a3cfe90'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:59.643503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a97b4d8e-38bf-4dd4-b119-68592a3cfe90 admin admin] The token cache is being invalidate because project 4a1093da0f6d4ce69499a561d173fcbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:13:59.644068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a97b4d8e-38bf-4dd4-b119-68592a3cfe90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:59.645463 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a97b4d8e-38bf-4dd4-b119-68592a3cfe90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:13:59.650594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 957/1901] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:13:59 2022] DELETE /identity/v3/projects/4a1093da0f6d4ce69499a561d173fcbd => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:13:59.655026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57758c3-b482-47e2-a23d-366830dbc3b8 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:59.655871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57758c3-b482-47e2-a23d-366830dbc3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:59.657012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57758c3-b482-47e2-a23d-366830dbc3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:59.689374 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-a57758c3-b482-47e2-a23d-366830dbc3b8 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:59.690965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a57758c3-b482-47e2-a23d-366830dbc3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:59.812106 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 945/1902] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:13:59.821984 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfebf06-7b5a-481a-a340-6f777e254dd2 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:59.822444 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfebf06-7b5a-481a-a340-6f777e254dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:59.822841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfebf06-7b5a-481a-a340-6f777e254dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:59.855409 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-bdfebf06-7b5a-481a-a340-6f777e254dd2 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:59.857331 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-bdfebf06-7b5a-481a-a340-6f777e254dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:59.867800 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 958/1903] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:13:59.873845 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57758c3-b482-47e2-a23d-366830dbc3b8 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:59.874022 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfebf06-7b5a-481a-a340-6f777e254dd2 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:59.876025 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a57758c3-b482-47e2-a23d-366830dbc3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:59.895104 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfebf06-7b5a-481a-a340-6f777e254dd2 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:59.905335 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9811eb-e10c-45af-be73-3b90ec9ceb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:59.906446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9811eb-e10c-45af-be73-3b90ec9ceb56 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:59.907874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9811eb-e10c-45af-be73-3b90ec9ceb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:59.908648 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9811eb-e10c-45af-be73-3b90ec9ceb56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:59.910013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9811eb-e10c-45af-be73-3b90ec9ceb56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:59.910190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57758c3-b482-47e2-a23d-366830dbc3b8 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:59.910495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9811eb-e10c-45af-be73-3b90ec9ceb56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:59.912999 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9811eb-e10c-45af-be73-3b90ec9ceb56 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:59.919334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34798881-f2e4-4a66-9826-e9a2d0ee7d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:59.920049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34798881-f2e4-4a66-9826-e9a2d0ee7d41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:59.920344 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34798881-f2e4-4a66-9826-e9a2d0ee7d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:59.920650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34798881-f2e4-4a66-9826-e9a2d0ee7d41 admin admin] PATH_INFO: `/v3/projects/b03c6bfc69444050a915366fb4e2dfc2` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:59.921941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34798881-f2e4-4a66-9826-e9a2d0ee7d41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b03c6bfc69444050a915366fb4e2dfc2)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:59.923931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34798881-f2e4-4a66-9826-e9a2d0ee7d41 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:59.932452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 959/1904] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:59.938336 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9811eb-e10c-45af-be73-3b90ec9ceb56 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:59.956380 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9811eb-e10c-45af-be73-3b90ec9ceb56 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:59.964396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06388b-18b3-4fb6-84ea-dabcc4626db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:59.965359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06388b-18b3-4fb6-84ea-dabcc4626db6 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:59.965690 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06388b-18b3-4fb6-84ea-dabcc4626db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:59.966462 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06388b-18b3-4fb6-84ea-dabcc4626db6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:59.967429 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06388b-18b3-4fb6-84ea-dabcc4626db6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:59.970962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06388b-18b3-4fb6-84ea-dabcc4626db6 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:00.019548 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-34798881-f2e4-4a66-9826-e9a2d0ee7d41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b03c6bfc69444050a915366fb4e2dfc2', 'request_id': 'req-34798881-f2e4-4a66-9826-e9a2d0ee7d41'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:00.028176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-34798881-f2e4-4a66-9826-e9a2d0ee7d41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b03c6bfc69444050a915366fb4e2dfc2', 'request_id': 'req-34798881-f2e4-4a66-9826-e9a2d0ee7d41'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:00.039887 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 960/1905] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:13:59 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:14:00.041702 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-34798881-f2e4-4a66-9826-e9a2d0ee7d41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b03c6bfc69444050a915366fb4e2dfc2', 'request_id': 'req-34798881-f2e4-4a66-9826-e9a2d0ee7d41'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:00.049219 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06388b-18b3-4fb6-84ea-dabcc4626db6 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:00.063726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-34798881-f2e4-4a66-9826-e9a2d0ee7d41 admin admin] The token cache is being invalidate because project b03c6bfc69444050a915366fb4e2dfc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:00.065200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-34798881-f2e4-4a66-9826-e9a2d0ee7d41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:00.066330 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-34798881-f2e4-4a66-9826-e9a2d0ee7d41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:00.068966 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 946/1906] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:13:59 2022] DELETE /identity/v3/projects/b03c6bfc69444050a915366fb4e2dfc2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:00.076184 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd707d55-e3e8-4ef1-b165-efd70e013e9f None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:00.078831 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd707d55-e3e8-4ef1-b165-efd70e013e9f None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:00.079325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd707d55-e3e8-4ef1-b165-efd70e013e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:00.157816 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-bd707d55-e3e8-4ef1-b165-efd70e013e9f None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:00.159123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-bd707d55-e3e8-4ef1-b165-efd70e013e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:00.194153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:00.224065 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06388b-18b3-4fb6-84ea-dabcc4626db6 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:00.239835 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645be11-1a98-47b6-aa5d-d4104c84965f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:00.245564 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645be11-1a98-47b6-aa5d-d4104c84965f admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:00.246470 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645be11-1a98-47b6-aa5d-d4104c84965f admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:00.247255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645be11-1a98-47b6-aa5d-d4104c84965f admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:00.265058 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645be11-1a98-47b6-aa5d-d4104c84965f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:00.270146 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645be11-1a98-47b6-aa5d-d4104c84965f admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:00.306886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 947/1907] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:00.309048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 961/1908] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:00 2022] POST /identity/v3/users => generated 283 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:00.311547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd707d55-e3e8-4ef1-b165-efd70e013e9f None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:00.313410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-bd707d55-e3e8-4ef1-b165-efd70e013e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:00.319932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0321e7c-d996-499b-8580-2764aca4dbd4 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:00.321142 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0321e7c-d996-499b-8580-2764aca4dbd4 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:00.322443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0321e7c-d996-499b-8580-2764aca4dbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:00.330419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd707d55-e3e8-4ef1-b165-efd70e013e9f None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:00.336675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc429fc-e533-43f9-8298-bcb07614187d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:00.338294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc429fc-e533-43f9-8298-bcb07614187d admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:00.338734 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc429fc-e533-43f9-8298-bcb07614187d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:00.339198 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc429fc-e533-43f9-8298-bcb07614187d admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:00.340559 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc429fc-e533-43f9-8298-bcb07614187d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:00.342780 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc429fc-e533-43f9-8298-bcb07614187d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:00.375902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-e0321e7c-d996-499b-8580-2764aca4dbd4 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:00.376637 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 948/1909] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:14:00.377881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc429fc-e533-43f9-8298-bcb07614187d admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:00.378825 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e0321e7c-d996-499b-8580-2764aca4dbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:00.405318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc429fc-e533-43f9-8298-bcb07614187d admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:00.413818 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6f3d5-13d3-4981-8b4f-2c6045a7ebae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:00.414791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 962/1910] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:00.415503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6f3d5-13d3-4981-8b4f-2c6045a7ebae admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:00.416294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6f3d5-13d3-4981-8b4f-2c6045a7ebae admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:00.416890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6f3d5-13d3-4981-8b4f-2c6045a7ebae admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:00.417837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6f3d5-13d3-4981-8b4f-2c6045a7ebae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:00.420347 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6f3d5-13d3-4981-8b4f-2c6045a7ebae admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:00.420552 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0321e7c-d996-499b-8580-2764aca4dbd4 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:00.437390 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 949/1911] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:00.442778 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6f3d5-13d3-4981-8b4f-2c6045a7ebae admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:00.460264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0321e7c-d996-499b-8580-2764aca4dbd4 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:00.463908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6f3d5-13d3-4981-8b4f-2c6045a7ebae admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:00.467618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb7dbc-603b-42cf-91f0-1c8811828dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:00.468636 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb7dbc-603b-42cf-91f0-1c8811828dbb admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:00.469207 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb7dbc-603b-42cf-91f0-1c8811828dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:00.469788 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb7dbc-603b-42cf-91f0-1c8811828dbb admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:00.471068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb7dbc-603b-42cf-91f0-1c8811828dbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:00.471454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb7dbc-603b-42cf-91f0-1c8811828dbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:00.473979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb7dbc-603b-42cf-91f0-1c8811828dbb admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:00.485995 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b180906-7cb5-4c65-90b2-81a72660243a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:00.487203 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b180906-7cb5-4c65-90b2-81a72660243a admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:00.488580 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b180906-7cb5-4c65-90b2-81a72660243a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:00.494224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b180906-7cb5-4c65-90b2-81a72660243a admin admin] PATH_INFO: `/v3/projects/04c0d8f33c144f23a34bbd7221cb7370/users/8f8cb1e8e2d5423f8be260c9e8863c1a/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:00.524636 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 963/1912] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:00.524846 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb7dbc-603b-42cf-91f0-1c8811828dbb admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:00.538672 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb7dbc-603b-42cf-91f0-1c8811828dbb admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:00.541885 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b180906-7cb5-4c65-90b2-81a72660243a admin admin] RBAC: Authorizing `identity:create_grant(project_id=04c0d8f33c144f23a34bbd7221cb7370, user_id=8f8cb1e8e2d5423f8be260c9e8863c1a, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:00.546028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14272f-79f6-4a59-beb7-819ef608e2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:00.547168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14272f-79f6-4a59-beb7-819ef608e2eb admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:00.547456 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14272f-79f6-4a59-beb7-819ef608e2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:00.547736 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14272f-79f6-4a59-beb7-819ef608e2eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:00.548644 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14272f-79f6-4a59-beb7-819ef608e2eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:00.551295 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14272f-79f6-4a59-beb7-819ef608e2eb admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:00.552179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b180906-7cb5-4c65-90b2-81a72660243a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:00.591259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 950/1913] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:00 2022] PUT /identity/v3/projects/04c0d8f33c144f23a34bbd7221cb7370/users/8f8cb1e8e2d5423f8be260c9e8863c1a/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:14:00.612356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712fd5d-2774-4c57-9657-c3144986b1ed None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:00.612356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712fd5d-2774-4c57-9657-c3144986b1ed None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:00.612356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712fd5d-2774-4c57-9657-c3144986b1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:00.626742 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 964/1914] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:00 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:14:00.632655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14272f-79f6-4a59-beb7-819ef608e2eb admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:00.725507 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-7712fd5d-2774-4c57-9657-c3144986b1ed None None] MFA Rules not processed for user `8f8cb1e8e2d5423f8be260c9e8863c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:00.729723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-7712fd5d-2774-4c57-9657-c3144986b1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:00.745297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:00.846578 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14272f-79f6-4a59-beb7-819ef608e2eb admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:00.855439 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa63015-46dd-470c-aec6-659e8b04a79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:00.856657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa63015-46dd-470c-aec6-659e8b04a79a admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:00.857236 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa63015-46dd-470c-aec6-659e8b04a79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:00.857791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa63015-46dd-470c-aec6-659e8b04a79a admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:00.859624 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa63015-46dd-470c-aec6-659e8b04a79a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:00.862631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa63015-46dd-470c-aec6-659e8b04a79a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:00.928323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 965/1915] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:00 2022] POST /identity/v3/users => generated 292 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:00.930121 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa63015-46dd-470c-aec6-659e8b04a79a admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:00.941679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 951/1916] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:00 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:00.949785 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712fd5d-2774-4c57-9657-c3144986b1ed None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:00.960651 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa63015-46dd-470c-aec6-659e8b04a79a admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:00.978661 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4f4f8-4c26-4647-a7f5-b82c8355d815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:00.979801 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4f4f8-4c26-4647-a7f5-b82c8355d815 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:00.980762 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4f4f8-4c26-4647-a7f5-b82c8355d815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:00.981170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4f4f8-4c26-4647-a7f5-b82c8355d815 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:00.982418 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4f4f8-4c26-4647-a7f5-b82c8355d815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:00.986542 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4f4f8-4c26-4647-a7f5-b82c8355d815 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:01.003124 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712fd5d-2774-4c57-9657-c3144986b1ed None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:01.011531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 966/1917] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:01.012756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4f4f8-4c26-4647-a7f5-b82c8355d815 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:01.013953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd86d13-07d0-4439-bf2f-b1ae81974853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:01.015609 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd86d13-07d0-4439-bf2f-b1ae81974853 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:01.016401 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd86d13-07d0-4439-bf2f-b1ae81974853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:01.017163 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd86d13-07d0-4439-bf2f-b1ae81974853 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:01.018577 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd86d13-07d0-4439-bf2f-b1ae81974853 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:01.025626 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd86d13-07d0-4439-bf2f-b1ae81974853 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:01.030214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4f4f8-4c26-4647-a7f5-b82c8355d815 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:01.037227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013364f3-3cf9-4dc0-a713-c52aa4a74e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:01.037883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013364f3-3cf9-4dc0-a713-c52aa4a74e0b admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:01.038281 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013364f3-3cf9-4dc0-a713-c52aa4a74e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:01.038562 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013364f3-3cf9-4dc0-a713-c52aa4a74e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:01.039316 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013364f3-3cf9-4dc0-a713-c52aa4a74e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:01.044531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013364f3-3cf9-4dc0-a713-c52aa4a74e0b admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:01.070848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 967/1918] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:01.079380 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013364f3-3cf9-4dc0-a713-c52aa4a74e0b admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:01.106129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013364f3-3cf9-4dc0-a713-c52aa4a74e0b admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:01.124153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500180a8-b13f-4040-9fe8-1047db627561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:01.126990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500180a8-b13f-4040-9fe8-1047db627561 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:01.127629 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500180a8-b13f-4040-9fe8-1047db627561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:01.128121 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500180a8-b13f-4040-9fe8-1047db627561 admin admin] PATH_INFO: `/v3/projects/03ff255ce0464ad49313432927bb4daf/users/16f05877e10442d2a8938f6805eff842/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:01.145522 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 952/1919] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:00 2022] POST /identity/v3/projects => generated 363 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:01.148037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd86d13-07d0-4439-bf2f-b1ae81974853 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:01.194859 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500180a8-b13f-4040-9fe8-1047db627561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03ff255ce0464ad49313432927bb4daf, user_id=16f05877e10442d2a8938f6805eff842, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:01.211728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500180a8-b13f-4040-9fe8-1047db627561 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:01.258320 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:01.339692 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 968/1920] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:01 2022] PUT /identity/v3/projects/03ff255ce0464ad49313432927bb4daf/users/16f05877e10442d2a8938f6805eff842/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:01.353649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554059a-c846-4de6-94ac-652c85256c3e None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:01.353649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554059a-c846-4de6-94ac-652c85256c3e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:01.358213 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554059a-c846-4de6-94ac-652c85256c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:01.381768 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd86d13-07d0-4439-bf2f-b1ae81974853 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:01.445473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-c554059a-c846-4de6-94ac-652c85256c3e None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:01.448034 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-c554059a-c846-4de6-94ac-652c85256c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:01.550438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7592347-7d1e-4b92-9680-a55b56e206b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:01.551758 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7592347-7d1e-4b92-9680-a55b56e206b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:01.552203 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7592347-7d1e-4b92-9680-a55b56e206b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:01.552742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7592347-7d1e-4b92-9680-a55b56e206b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:01.555093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7592347-7d1e-4b92-9680-a55b56e206b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:01.559086 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7592347-7d1e-4b92-9680-a55b56e206b5 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:01.609684 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 953/1921] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:01 2022] POST /identity/v3/users => generated 288 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:14:01.614840 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7592347-7d1e-4b92-9680-a55b56e206b5 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:01.632011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 969/1922] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:01.633259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7592347-7d1e-4b92-9680-a55b56e206b5 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:01.639940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c0687-590d-4e79-8efc-308cdf3d8489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:01.641224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c0687-590d-4e79-8efc-308cdf3d8489 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:01.641224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c0687-590d-4e79-8efc-308cdf3d8489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:01.641522 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c0687-590d-4e79-8efc-308cdf3d8489 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:01.642877 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c0687-590d-4e79-8efc-308cdf3d8489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:01.646814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c0687-590d-4e79-8efc-308cdf3d8489 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:01.648247 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554059a-c846-4de6-94ac-652c85256c3e None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:01.652369 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-c554059a-c846-4de6-94ac-652c85256c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:01.663178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 954/1923] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:14:01.668211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c0687-590d-4e79-8efc-308cdf3d8489 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:01.680415 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c0687-590d-4e79-8efc-308cdf3d8489 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:01.686351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1beacad-40bc-4786-b8f2-82a55f14c553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:01.687510 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beacad-40bc-4786-b8f2-82a55f14c553 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:01.687803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beacad-40bc-4786-b8f2-82a55f14c553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:01.688170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beacad-40bc-4786-b8f2-82a55f14c553 admin admin] PATH_INFO: `/v3/projects/249024e12a8c41d3b831203719464e89/users/26b3db3a221f45e4a69b13fa64f52e7e/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:01.723803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1beacad-40bc-4786-b8f2-82a55f14c553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=249024e12a8c41d3b831203719464e89, user_id=26b3db3a221f45e4a69b13fa64f52e7e, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:01.730147 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1beacad-40bc-4786-b8f2-82a55f14c553 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:01.744641 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554059a-c846-4de6-94ac-652c85256c3e None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:01.750382 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda084a-1ac8-492a-95ae-9b22aefc5cd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:01.751202 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda084a-1ac8-492a-95ae-9b22aefc5cd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:01.751636 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda084a-1ac8-492a-95ae-9b22aefc5cd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:01.752105 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda084a-1ac8-492a-95ae-9b22aefc5cd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:01.761519 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 955/1924] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:01 2022] PUT /identity/v3/projects/249024e12a8c41d3b831203719464e89/users/26b3db3a221f45e4a69b13fa64f52e7e/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:14:01.768349 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1beacad-40bc-4786-b8f2-82a55f14c553 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:01.769905 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda084a-1ac8-492a-95ae-9b22aefc5cd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:01.771920 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda084a-1ac8-492a-95ae-9b22aefc5cd2 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:01.789688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 970/1925] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:01.941667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1beacad-40bc-4786-b8f2-82a55f14c553 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:01.950591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08d2fc-d6b8-48cb-8ebe-04241ef9433a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:01.951934 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08d2fc-d6b8-48cb-8ebe-04241ef9433a admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:01.951934 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08d2fc-d6b8-48cb-8ebe-04241ef9433a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:01.952293 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08d2fc-d6b8-48cb-8ebe-04241ef9433a admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:01.953235 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08d2fc-d6b8-48cb-8ebe-04241ef9433a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:01.955305 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08d2fc-d6b8-48cb-8ebe-04241ef9433a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:01.974436 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 956/1926] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:14:01.977076 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda084a-1ac8-492a-95ae-9b22aefc5cd2 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:02.012880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda084a-1ac8-492a-95ae-9b22aefc5cd2 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:02.046231 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e95fc2-0383-470e-b9da-d95b6cc963c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:02.047332 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e95fc2-0383-470e-b9da-d95b6cc963c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:02.047332 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e95fc2-0383-470e-b9da-d95b6cc963c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:02.047794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e95fc2-0383-470e-b9da-d95b6cc963c2 admin admin] PATH_INFO: `/v3/projects/249024e12a8c41d3b831203719464e89/users/26b3db3a221f45e4a69b13fa64f52e7e/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:02.057969 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e95fc2-0383-470e-b9da-d95b6cc963c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=249024e12a8c41d3b831203719464e89, user_id=26b3db3a221f45e4a69b13fa64f52e7e, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:02.063781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e95fc2-0383-470e-b9da-d95b6cc963c2 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:02.084519 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:02.126866 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 971/1927] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:01 2022] PUT /identity/v3/projects/249024e12a8c41d3b831203719464e89/users/26b3db3a221f45e4a69b13fa64f52e7e/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:02.129907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd43df5-e9d2-4464-b744-f5beabee5cb7 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:02.130741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd43df5-e9d2-4464-b744-f5beabee5cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:02.131298 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd43df5-e9d2-4464-b744-f5beabee5cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:02.166105 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd83d9-254b-4e1a-a5c9-b10b428316e5 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:02.167003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd83d9-254b-4e1a-a5c9-b10b428316e5 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:02.167874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd83d9-254b-4e1a-a5c9-b10b428316e5 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:02.172949 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 972/1928] 10.222.0.93 () {54 vars in 921 bytes} [Mon Aug 1 15:14:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:14:02.179248 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-6fd43df5-e9d2-4464-b744-f5beabee5cb7 None None] MFA Rules not processed for user `26b3db3a221f45e4a69b13fa64f52e7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:02.181714 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-6fd43df5-e9d2-4464-b744-f5beabee5cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:02.229439 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed9ef0-48a5-4586-ac08-d6ad178c9042 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:02.230061 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed9ef0-48a5-4586-ac08-d6ad178c9042 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:02.230883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed9ef0-48a5-4586-ac08-d6ad178c9042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:02.270717 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:02.346877 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-70ed9ef0-48a5-4586-ac08-d6ad178c9042 None None] MFA Rules not processed for user `a07e3e25e6a8466fb237dcb836aecda2`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:02.353579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-70ed9ef0-48a5-4586-ac08-d6ad178c9042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:02.471417 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 957/1929] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:02 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:02.485724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd43df5-e9d2-4464-b744-f5beabee5cb7 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:02.487487 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-6fd43df5-e9d2-4464-b744-f5beabee5cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:02.600880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 973/1930] 10.222.0.93 () {58 vars in 1018 bytes} [Mon Aug 1 15:14:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:02.622173 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed9ef0-48a5-4586-ac08-d6ad178c9042 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:02.623813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-70ed9ef0-48a5-4586-ac08-d6ad178c9042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:02.639501 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed9ef0-48a5-4586-ac08-d6ad178c9042 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:02.669170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475b80b-2319-4504-8102-1d42133b74b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07e3e25e6a8466fb237dcb836aecda2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:02.669895 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475b80b-2319-4504-8102-1d42133b74b7 service placement] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:02.670174 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475b80b-2319-4504-8102-1d42133b74b7 service placement] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:02.672021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475b80b-2319-4504-8102-1d42133b74b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:02.692972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd43df5-e9d2-4464-b744-f5beabee5cb7 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:02.695488 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475b80b-2319-4504-8102-1d42133b74b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:02.702816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475b80b-2319-4504-8102-1d42133b74b7 service placement] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:02.708366 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307b2eb-6c50-4907-9a86-139161d51cb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:02.729684 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307b2eb-6c50-4907-9a86-139161d51cb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:02.730400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307b2eb-6c50-4907-9a86-139161d51cb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:02.731135 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307b2eb-6c50-4907-9a86-139161d51cb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:02.766948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 974/1931] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:14:02.769394 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307b2eb-6c50-4907-9a86-139161d51cb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:02.771283 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307b2eb-6c50-4907-9a86-139161d51cb7 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:02.809513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 958/1932] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:02 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:03.000603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475b80b-2319-4504-8102-1d42133b74b7 service placement] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:03.032090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475b80b-2319-4504-8102-1d42133b74b7 service placement] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:03.054110 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133c9e0-2718-4cee-b914-4d1efdf3acd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:03.057007 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133c9e0-2718-4cee-b914-4d1efdf3acd7 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:03.057007 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133c9e0-2718-4cee-b914-4d1efdf3acd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:03.057433 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133c9e0-2718-4cee-b914-4d1efdf3acd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:03.223111 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133c9e0-2718-4cee-b914-4d1efdf3acd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:03.225825 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133c9e0-2718-4cee-b914-4d1efdf3acd7 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:03.244297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 975/1933] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:02 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:05.234678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307b2eb-6c50-4907-9a86-139161d51cb7 service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:05.238973 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d307b2eb-6c50-4907-9a86-139161d51cb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:05.590049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307b2eb-6c50-4907-9a86-139161d51cb7 service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:05.609794 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835bd34-014a-4e4b-858a-e293a8e655ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:05.613504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835bd34-014a-4e4b-858a-e293a8e655ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:05.613876 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835bd34-014a-4e4b-858a-e293a8e655ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:05.614231 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835bd34-014a-4e4b-858a-e293a8e655ca admin admin] PATH_INFO: `/v3/users/14611c117623417c99ab1a376f82681b` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:05.616714 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835bd34-014a-4e4b-858a-e293a8e655ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=14611c117623417c99ab1a376f82681b)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:05.622061 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835bd34-014a-4e4b-858a-e293a8e655ca admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:05.853715 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3835bd34-014a-4e4b-858a-e293a8e655ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14611c117623417c99ab1a376f82681b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:05.855058 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3835bd34-014a-4e4b-858a-e293a8e655ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14611c117623417c99ab1a376f82681b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:05.884199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3835bd34-014a-4e4b-858a-e293a8e655ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14611c117623417c99ab1a376f82681b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:05.895042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3835bd34-014a-4e4b-858a-e293a8e655ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14611c117623417c99ab1a376f82681b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:05.925745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:05.930476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3835bd34-014a-4e4b-858a-e293a8e655ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14611c117623417c99ab1a376f82681b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:05.967258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 959/1934] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:14:05 2022] DELETE /identity/v3/users/14611c117623417c99ab1a376f82681b => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:06.475229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133c9e0-2718-4cee-b914-4d1efdf3acd7 service cinder] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:06.513573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133c9e0-2718-4cee-b914-4d1efdf3acd7 service cinder] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:06.809183 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9003cc3-0baf-4e7f-9bb0-973a82799188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:06.817790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9003cc3-0baf-4e7f-9bb0-973a82799188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:06.817790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9003cc3-0baf-4e7f-9bb0-973a82799188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:06.817790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9003cc3-0baf-4e7f-9bb0-973a82799188 admin admin] PATH_INFO: `/v3/projects/d51ad2cf95f9437bbfc591999c7a4aa8` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:06.820279 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9003cc3-0baf-4e7f-9bb0-973a82799188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d51ad2cf95f9437bbfc591999c7a4aa8)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:06.834117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9003cc3-0baf-4e7f-9bb0-973a82799188 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:06.942603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:07.073875 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a9003cc3-0baf-4e7f-9bb0-973a82799188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd51ad2cf95f9437bbfc591999c7a4aa8', 'request_id': 'req-a9003cc3-0baf-4e7f-9bb0-973a82799188'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:07.116896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a9003cc3-0baf-4e7f-9bb0-973a82799188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd51ad2cf95f9437bbfc591999c7a4aa8', 'request_id': 'req-a9003cc3-0baf-4e7f-9bb0-973a82799188'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:07.138449 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a9003cc3-0baf-4e7f-9bb0-973a82799188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd51ad2cf95f9437bbfc591999c7a4aa8', 'request_id': 'req-a9003cc3-0baf-4e7f-9bb0-973a82799188'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:07.171620 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a9003cc3-0baf-4e7f-9bb0-973a82799188 admin admin] The token cache is being invalidate because project d51ad2cf95f9437bbfc591999c7a4aa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:07.171914 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a9003cc3-0baf-4e7f-9bb0-973a82799188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:07.176698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a9003cc3-0baf-4e7f-9bb0-973a82799188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:07.179402 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 976/1935] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:14:06 2022] DELETE /identity/v3/projects/d51ad2cf95f9437bbfc591999c7a4aa8 => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:07.185559 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835bd34-014a-4e4b-858a-e293a8e655ca admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:07.189210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-3835bd34-014a-4e4b-858a-e293a8e655ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:07.244370 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835bd34-014a-4e4b-858a-e293a8e655ca admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:07.300275 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9003cc3-0baf-4e7f-9bb0-973a82799188 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:07.308915 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a9003cc3-0baf-4e7f-9bb0-973a82799188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:07.414535 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad3ad9-1aa4-4e7e-97ad-ffaefe360387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:07.416371 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad3ad9-1aa4-4e7e-97ad-ffaefe360387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:07.416933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad3ad9-1aa4-4e7e-97ad-ffaefe360387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:07.417325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad3ad9-1aa4-4e7e-97ad-ffaefe360387 admin admin] PATH_INFO: `/v3/users/8d8447d902a34ba8becb8ba5e208a715` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:07.418876 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad3ad9-1aa4-4e7e-97ad-ffaefe360387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d8447d902a34ba8becb8ba5e208a715)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:07.421257 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad3ad9-1aa4-4e7e-97ad-ffaefe360387 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:07.541275 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9003cc3-0baf-4e7f-9bb0-973a82799188 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:07.554850 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-48ad3ad9-1aa4-4e7e-97ad-ffaefe360387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d8447d902a34ba8becb8ba5e208a715'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:07.556951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-48ad3ad9-1aa4-4e7e-97ad-ffaefe360387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d8447d902a34ba8becb8ba5e208a715'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:07.557166 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423b5e5-90f2-4cd2-9ace-f0bc608a9269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:07.558444 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423b5e5-90f2-4cd2-9ace-f0bc608a9269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:07.558800 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423b5e5-90f2-4cd2-9ace-f0bc608a9269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:07.559141 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423b5e5-90f2-4cd2-9ace-f0bc608a9269 admin admin] PATH_INFO: `/v3/users/8f8cb1e8e2d5423f8be260c9e8863c1a` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:07.561107 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423b5e5-90f2-4cd2-9ace-f0bc608a9269 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f8cb1e8e2d5423f8be260c9e8863c1a)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:07.572961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423b5e5-90f2-4cd2-9ace-f0bc608a9269 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:07.574003 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-48ad3ad9-1aa4-4e7e-97ad-ffaefe360387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d8447d902a34ba8becb8ba5e208a715'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:07.577319 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-48ad3ad9-1aa4-4e7e-97ad-ffaefe360387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d8447d902a34ba8becb8ba5e208a715'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:07.650895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-48ad3ad9-1aa4-4e7e-97ad-ffaefe360387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d8447d902a34ba8becb8ba5e208a715'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:07.693356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 960/1936] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:14:07 2022] DELETE /identity/v3/users/8d8447d902a34ba8becb8ba5e208a715 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:07.742496 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b423b5e5-90f2-4cd2-9ace-f0bc608a9269 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f8cb1e8e2d5423f8be260c9e8863c1a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:07.770412 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b423b5e5-90f2-4cd2-9ace-f0bc608a9269 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f8cb1e8e2d5423f8be260c9e8863c1a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:07.789520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b423b5e5-90f2-4cd2-9ace-f0bc608a9269 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f8cb1e8e2d5423f8be260c9e8863c1a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:07.791084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b423b5e5-90f2-4cd2-9ace-f0bc608a9269 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f8cb1e8e2d5423f8be260c9e8863c1a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:07.791519 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-b423b5e5-90f2-4cd2-9ace-f0bc608a9269 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f8cb1e8e2d5423f8be260c9e8863c1a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:07.867489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 977/1937] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:14:07 2022] DELETE /identity/v3/users/8f8cb1e8e2d5423f8be260c9e8863c1a => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:07.873997 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab03df-787f-48a3-98a5-7c0c77ec37b8 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:07.874285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab03df-787f-48a3-98a5-7c0c77ec37b8 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:07.874650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab03df-787f-48a3-98a5-7c0c77ec37b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:07.949765 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-7bab03df-787f-48a3-98a5-7c0c77ec37b8 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:07.956779 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-7bab03df-787f-48a3-98a5-7c0c77ec37b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:08.123705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 961/1938] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:08.241208 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423b5e5-90f2-4cd2-9ace-f0bc608a9269 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:08.244079 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b423b5e5-90f2-4cd2-9ace-f0bc608a9269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:08.342025 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:08.414771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423b5e5-90f2-4cd2-9ace-f0bc608a9269 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:08.427882 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469697e-38f9-43bb-b80d-aea3e3ff6825 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:08.428739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469697e-38f9-43bb-b80d-aea3e3ff6825 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:08.429183 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469697e-38f9-43bb-b80d-aea3e3ff6825 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:08.429468 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469697e-38f9-43bb-b80d-aea3e3ff6825 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:08.459797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469697e-38f9-43bb-b80d-aea3e3ff6825 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:08.461527 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469697e-38f9-43bb-b80d-aea3e3ff6825 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:08.492550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 978/1939] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:08.659532 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab03df-787f-48a3-98a5-7c0c77ec37b8 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:08.665194 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-7bab03df-787f-48a3-98a5-7c0c77ec37b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:08.709307 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:08.722288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab03df-787f-48a3-98a5-7c0c77ec37b8 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:08.753870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb6c86-b978-458b-b633-cecb29be53cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:08.754933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb6c86-b978-458b-b633-cecb29be53cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:08.755284 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb6c86-b978-458b-b633-cecb29be53cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:08.755572 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb6c86-b978-458b-b633-cecb29be53cd admin admin] PATH_INFO: `/v3/projects/0aa2bf6eaed047b99cf3b2758cda865a` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:08.757969 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb6c86-b978-458b-b633-cecb29be53cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aa2bf6eaed047b99cf3b2758cda865a)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:08.761493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb6c86-b978-458b-b633-cecb29be53cd admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:08.873107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1ecb6c86-b978-458b-b633-cecb29be53cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aa2bf6eaed047b99cf3b2758cda865a', 'request_id': 'req-1ecb6c86-b978-458b-b633-cecb29be53cd'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:08.895158 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1ecb6c86-b978-458b-b633-cecb29be53cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aa2bf6eaed047b99cf3b2758cda865a', 'request_id': 'req-1ecb6c86-b978-458b-b633-cecb29be53cd'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:08.909663 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1ecb6c86-b978-458b-b633-cecb29be53cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aa2bf6eaed047b99cf3b2758cda865a', 'request_id': 'req-1ecb6c86-b978-458b-b633-cecb29be53cd'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:08.930764 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1ecb6c86-b978-458b-b633-cecb29be53cd admin admin] The token cache is being invalidate because project 0aa2bf6eaed047b99cf3b2758cda865a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:08.931439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1ecb6c86-b978-458b-b633-cecb29be53cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:08.932791 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1ecb6c86-b978-458b-b633-cecb29be53cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:08.935520 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 962/1940] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:14:08 2022] DELETE /identity/v3/projects/0aa2bf6eaed047b99cf3b2758cda865a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:08.943163 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f2ab0-6484-4f35-b041-5f1dbf19e216 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:08.943811 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f2ab0-6484-4f35-b041-5f1dbf19e216 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:08.944422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f2ab0-6484-4f35-b041-5f1dbf19e216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:08.988891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-fe7f2ab0-6484-4f35-b041-5f1dbf19e216 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:08.991401 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-fe7f2ab0-6484-4f35-b041-5f1dbf19e216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:09.236748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 979/1941] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:09.246823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a89084-668a-467a-9a33-7ac98f830770 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:09.247152 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a89084-668a-467a-9a33-7ac98f830770 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:09.247628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a89084-668a-467a-9a33-7ac98f830770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:09.287442 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-e2a89084-668a-467a-9a33-7ac98f830770 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:09.296809 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e2a89084-668a-467a-9a33-7ac98f830770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:09.314843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 963/1942] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:09.322709 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f2ab0-6484-4f35-b041-5f1dbf19e216 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:09.353079 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f2ab0-6484-4f35-b041-5f1dbf19e216 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:09.363487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e45c7b-bf3c-4ba0-ac38-f5a02666b543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:09.365575 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e45c7b-bf3c-4ba0-ac38-f5a02666b543 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:09.366735 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e45c7b-bf3c-4ba0-ac38-f5a02666b543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:09.367279 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e45c7b-bf3c-4ba0-ac38-f5a02666b543 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:09.369326 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e45c7b-bf3c-4ba0-ac38-f5a02666b543 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:09.370659 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e45c7b-bf3c-4ba0-ac38-f5a02666b543 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:09.375812 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e45c7b-bf3c-4ba0-ac38-f5a02666b543 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:09.426350 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 980/1943] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:09.432830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a89084-668a-467a-9a33-7ac98f830770 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:09.445919 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a89084-668a-467a-9a33-7ac98f830770 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:09.466314 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2c35f-aeed-4dac-9c4e-dc22c6fdd0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:09.467786 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2c35f-aeed-4dac-9c4e-dc22c6fdd0f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:09.468241 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2c35f-aeed-4dac-9c4e-dc22c6fdd0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:09.468815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2c35f-aeed-4dac-9c4e-dc22c6fdd0f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:09.469738 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2c35f-aeed-4dac-9c4e-dc22c6fdd0f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:09.474182 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2c35f-aeed-4dac-9c4e-dc22c6fdd0f4 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:09.579066 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 964/1944] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:09 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:14:09.595824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e45c7b-bf3c-4ba0-ac38-f5a02666b543 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:09.608809 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e45c7b-bf3c-4ba0-ac38-f5a02666b543 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:09.722356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da9e4c-401e-4ebe-983b-8fb9aaf6a414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:09.723860 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da9e4c-401e-4ebe-983b-8fb9aaf6a414 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:09.724299 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da9e4c-401e-4ebe-983b-8fb9aaf6a414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:09.724939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da9e4c-401e-4ebe-983b-8fb9aaf6a414 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:09.725961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da9e4c-401e-4ebe-983b-8fb9aaf6a414 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:09.729714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da9e4c-401e-4ebe-983b-8fb9aaf6a414 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:09.773591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 981/1945] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:09 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:09.780675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2c35f-aeed-4dac-9c4e-dc22c6fdd0f4 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:09.804370 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2c35f-aeed-4dac-9c4e-dc22c6fdd0f4 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:09.814797 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ef8e2-abf4-4890-9ca5-708c1b2ae9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:09.816201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ef8e2-abf4-4890-9ca5-708c1b2ae9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:09.817553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ef8e2-abf4-4890-9ca5-708c1b2ae9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:09.817553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ef8e2-abf4-4890-9ca5-708c1b2ae9d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:09.818272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ef8e2-abf4-4890-9ca5-708c1b2ae9d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:09.825742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ef8e2-abf4-4890-9ca5-708c1b2ae9d8 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:09.853180 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 965/1946] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:14:09.859866 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da9e4c-401e-4ebe-983b-8fb9aaf6a414 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:09.896319 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da9e4c-401e-4ebe-983b-8fb9aaf6a414 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:09.915697 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6875fd20-5fa5-495d-886c-e5dc191b0d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:09.917067 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875fd20-5fa5-495d-886c-e5dc191b0d2c admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:09.918924 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875fd20-5fa5-495d-886c-e5dc191b0d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:09.920409 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875fd20-5fa5-495d-886c-e5dc191b0d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:09.925005 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6875fd20-5fa5-495d-886c-e5dc191b0d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:09.939923 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6875fd20-5fa5-495d-886c-e5dc191b0d2c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:09.996021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 982/1947] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:14:09.999168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ef8e2-abf4-4890-9ca5-708c1b2ae9d8 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:10.026916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ef8e2-abf4-4890-9ca5-708c1b2ae9d8 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:10.045050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee2441-19ab-4f74-9d2a-f6f36ddd3e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:10.045753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee2441-19ab-4f74-9d2a-f6f36ddd3e8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:10.046029 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee2441-19ab-4f74-9d2a-f6f36ddd3e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:10.046304 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee2441-19ab-4f74-9d2a-f6f36ddd3e8a admin admin] PATH_INFO: `/v3/projects/652cef8527db4764ad1eb4fbf60704b6/users/e3455625067a4924a2095f89a129d454/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:10.097796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee2441-19ab-4f74-9d2a-f6f36ddd3e8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=652cef8527db4764ad1eb4fbf60704b6, user_id=e3455625067a4924a2095f89a129d454, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:10.102090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee2441-19ab-4f74-9d2a-f6f36ddd3e8a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:10.145725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 966/1948] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:09 2022] PUT /identity/v3/projects/652cef8527db4764ad1eb4fbf60704b6/users/e3455625067a4924a2095f89a129d454/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:10.150455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d4959-72f3-4c4e-9460-06a1becb584c None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:10.150923 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d4959-72f3-4c4e-9460-06a1becb584c None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:10.151674 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d4959-72f3-4c4e-9460-06a1becb584c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:10.218342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-777d4959-72f3-4c4e-9460-06a1becb584c None None] MFA Rules not processed for user `e3455625067a4924a2095f89a129d454`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:10.219460 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-777d4959-72f3-4c4e-9460-06a1becb584c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:10.347390 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:10.560667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 983/1949] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:10.567534 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee2441-19ab-4f74-9d2a-f6f36ddd3e8a admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:10.759290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee2441-19ab-4f74-9d2a-f6f36ddd3e8a admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:10.768621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187d42b-6ecf-4282-bc3f-da41c8372d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:10.769306 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187d42b-6ecf-4282-bc3f-da41c8372d0c admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:10.769593 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187d42b-6ecf-4282-bc3f-da41c8372d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:10.769939 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187d42b-6ecf-4282-bc3f-da41c8372d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:10.772440 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187d42b-6ecf-4282-bc3f-da41c8372d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:10.777563 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187d42b-6ecf-4282-bc3f-da41c8372d0c admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:10.852048 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 967/1950] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:10 2022] POST /identity/v3/projects => generated 385 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:10.853415 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d4959-72f3-4c4e-9460-06a1becb584c None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:10.875994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d4959-72f3-4c4e-9460-06a1becb584c None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:11.079867 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63ad98-a09d-47a9-9b0f-206b2a324b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:11.081419 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63ad98-a09d-47a9-9b0f-206b2a324b45 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:11.081950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63ad98-a09d-47a9-9b0f-206b2a324b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:11.082462 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63ad98-a09d-47a9-9b0f-206b2a324b45 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:11.083398 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63ad98-a09d-47a9-9b0f-206b2a324b45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:11.085870 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63ad98-a09d-47a9-9b0f-206b2a324b45 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:11.119622 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 984/1951] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:10 2022] POST /identity/v3/users => generated 299 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:11.124005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187d42b-6ecf-4282-bc3f-da41c8372d0c admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:11.137913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187d42b-6ecf-4282-bc3f-da41c8372d0c admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:11.144092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28abf6-28c2-4112-8234-6522c7aa3c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:11.144805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28abf6-28c2-4112-8234-6522c7aa3c78 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:11.145086 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28abf6-28c2-4112-8234-6522c7aa3c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:11.145364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28abf6-28c2-4112-8234-6522c7aa3c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:11.146188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28abf6-28c2-4112-8234-6522c7aa3c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:11.148144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28abf6-28c2-4112-8234-6522c7aa3c78 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:11.151838 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:11.171819 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63ad98-a09d-47a9-9b0f-206b2a324b45 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:11.177694 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 968/1952] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:11.185429 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63ad98-a09d-47a9-9b0f-206b2a324b45 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:11.192696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a0999-1eae-4a6a-84ff-660139706907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:11.194185 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a0999-1eae-4a6a-84ff-660139706907 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:11.194667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a0999-1eae-4a6a-84ff-660139706907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:11.194963 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a0999-1eae-4a6a-84ff-660139706907 admin admin] PATH_INFO: `/v3/projects/900adb03cf9941c1be25fba21adcff7b/users/9f4b2ffdc8b44d8fbd15d967d9078a77/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:11.237329 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a0999-1eae-4a6a-84ff-660139706907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=900adb03cf9941c1be25fba21adcff7b, user_id=9f4b2ffdc8b44d8fbd15d967d9078a77, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:11.245607 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a0999-1eae-4a6a-84ff-660139706907 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:11.293342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 985/1953] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:11 2022] PUT /identity/v3/projects/900adb03cf9941c1be25fba21adcff7b/users/9f4b2ffdc8b44d8fbd15d967d9078a77/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:11.300483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28abf6-28c2-4112-8234-6522c7aa3c78 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:11.324847 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28abf6-28c2-4112-8234-6522c7aa3c78 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:11.487626 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c036759-5292-4d0f-8afc-e185a90f5151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:11.488137 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c036759-5292-4d0f-8afc-e185a90f5151 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:11.488586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c036759-5292-4d0f-8afc-e185a90f5151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:11.488965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c036759-5292-4d0f-8afc-e185a90f5151 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:11.499488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c036759-5292-4d0f-8afc-e185a90f5151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:11.511030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c036759-5292-4d0f-8afc-e185a90f5151 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:11.533330 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 969/1954] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:11.539278 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a0999-1eae-4a6a-84ff-660139706907 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:11.558100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a0999-1eae-4a6a-84ff-660139706907 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:11.568497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb404c7f-dfd0-4f01-a823-5f75853eae7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:11.570057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb404c7f-dfd0-4f01-a823-5f75853eae7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:11.570559 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb404c7f-dfd0-4f01-a823-5f75853eae7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:11.571090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb404c7f-dfd0-4f01-a823-5f75853eae7c admin admin] PATH_INFO: `/v3/projects/900adb03cf9941c1be25fba21adcff7b/users/9f4b2ffdc8b44d8fbd15d967d9078a77/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:11.575152 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb404c7f-dfd0-4f01-a823-5f75853eae7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=900adb03cf9941c1be25fba21adcff7b, user_id=9f4b2ffdc8b44d8fbd15d967d9078a77, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:11.581563 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb404c7f-dfd0-4f01-a823-5f75853eae7c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:11.615085 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 986/1955] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:11 2022] PUT /identity/v3/projects/900adb03cf9941c1be25fba21adcff7b/users/9f4b2ffdc8b44d8fbd15d967d9078a77/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:11.634356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d529a2-7509-42fe-a26c-e9c828ae2d78 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:11.634981 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d529a2-7509-42fe-a26c-e9c828ae2d78 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:11.635350 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d529a2-7509-42fe-a26c-e9c828ae2d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:11.681979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-a2d529a2-7509-42fe-a26c-e9c828ae2d78 None None] MFA Rules not processed for user `9f4b2ffdc8b44d8fbd15d967d9078a77`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:11.684531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a2d529a2-7509-42fe-a26c-e9c828ae2d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:11.902100 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 970/1956] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:11.924177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb404c7f-dfd0-4f01-a823-5f75853eae7c admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:11.930072 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-fb404c7f-dfd0-4f01-a823-5f75853eae7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:11.931874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d529a2-7509-42fe-a26c-e9c828ae2d78 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:11.939780 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a2d529a2-7509-42fe-a26c-e9c828ae2d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:12.148222 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb404c7f-dfd0-4f01-a823-5f75853eae7c admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:12.153360 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3036a84-775b-4eec-a51b-871e9ac4822f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:12.157606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3036a84-775b-4eec-a51b-871e9ac4822f service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:12.158251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3036a84-775b-4eec-a51b-871e9ac4822f service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:12.159074 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3036a84-775b-4eec-a51b-871e9ac4822f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:12.167448 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d529a2-7509-42fe-a26c-e9c828ae2d78 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:12.173567 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde045ce-7e2c-4713-9f4a-867790827dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:12.174783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde045ce-7e2c-4713-9f4a-867790827dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:12.174783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde045ce-7e2c-4713-9f4a-867790827dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:12.175044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde045ce-7e2c-4713-9f4a-867790827dc9 admin admin] PATH_INFO: `/v3/projects/04c0d8f33c144f23a34bbd7221cb7370` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:12.176456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde045ce-7e2c-4713-9f4a-867790827dc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04c0d8f33c144f23a34bbd7221cb7370)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:12.179144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde045ce-7e2c-4713-9f4a-867790827dc9 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:12.215814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:12.220305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:12.277602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3036a84-775b-4eec-a51b-871e9ac4822f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:12.296566 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-dde045ce-7e2c-4713-9f4a-867790827dc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04c0d8f33c144f23a34bbd7221cb7370', 'request_id': 'req-dde045ce-7e2c-4713-9f4a-867790827dc9'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:12.307711 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3036a84-775b-4eec-a51b-871e9ac4822f service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:12.317689 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-dde045ce-7e2c-4713-9f4a-867790827dc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04c0d8f33c144f23a34bbd7221cb7370', 'request_id': 'req-dde045ce-7e2c-4713-9f4a-867790827dc9'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:12.337354 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-dde045ce-7e2c-4713-9f4a-867790827dc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04c0d8f33c144f23a34bbd7221cb7370', 'request_id': 'req-dde045ce-7e2c-4713-9f4a-867790827dc9'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:12.338656 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 987/1957] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:12.368042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-dde045ce-7e2c-4713-9f4a-867790827dc9 admin admin] The token cache is being invalidate because project 04c0d8f33c144f23a34bbd7221cb7370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:12.369227 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-dde045ce-7e2c-4713-9f4a-867790827dc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:12.370736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-dde045ce-7e2c-4713-9f4a-867790827dc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:12.374968 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 971/1958] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:14:11 2022] DELETE /identity/v3/projects/04c0d8f33c144f23a34bbd7221cb7370 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:12.384892 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3036a84-775b-4eec-a51b-871e9ac4822f service cinder] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:12.409283 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3036a84-775b-4eec-a51b-871e9ac4822f service cinder] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:12.562289 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e0e9a-8ae4-453b-ba03-a3ac59b48dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:12.563779 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e0e9a-8ae4-453b-ba03-a3ac59b48dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:12.564546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e0e9a-8ae4-453b-ba03-a3ac59b48dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:12.565316 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e0e9a-8ae4-453b-ba03-a3ac59b48dad admin admin] PATH_INFO: `/v3/users/26b3db3a221f45e4a69b13fa64f52e7e` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:12.567867 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e0e9a-8ae4-453b-ba03-a3ac59b48dad admin admin] RBAC: Authorizing `identity:delete_user(user_id=26b3db3a221f45e4a69b13fa64f52e7e)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:12.573156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e0e9a-8ae4-453b-ba03-a3ac59b48dad admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:12.710789 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-869e0e9a-8ae4-453b-ba03-a3ac59b48dad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26b3db3a221f45e4a69b13fa64f52e7e'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:12.740505 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-869e0e9a-8ae4-453b-ba03-a3ac59b48dad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26b3db3a221f45e4a69b13fa64f52e7e'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:12.788539 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-869e0e9a-8ae4-453b-ba03-a3ac59b48dad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26b3db3a221f45e4a69b13fa64f52e7e'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:12.790176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-869e0e9a-8ae4-453b-ba03-a3ac59b48dad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26b3db3a221f45e4a69b13fa64f52e7e'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:12.791439 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-869e0e9a-8ae4-453b-ba03-a3ac59b48dad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26b3db3a221f45e4a69b13fa64f52e7e'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:12.864722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 988/1959] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:14:12 2022] DELETE /identity/v3/users/26b3db3a221f45e4a69b13fa64f52e7e => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:13.800743 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde045ce-7e2c-4713-9f4a-867790827dc9 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:13.803669 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-dde045ce-7e2c-4713-9f4a-867790827dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:13.838159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde045ce-7e2c-4713-9f4a-867790827dc9 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:14.069582 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a712a00-2b2a-4937-8469-dce20a226a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:14.075272 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a712a00-2b2a-4937-8469-dce20a226a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:14.076116 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a712a00-2b2a-4937-8469-dce20a226a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:14.087483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a712a00-2b2a-4937-8469-dce20a226a6d admin admin] PATH_INFO: `/v3/projects/249024e12a8c41d3b831203719464e89` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:14.090538 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a712a00-2b2a-4937-8469-dce20a226a6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=249024e12a8c41d3b831203719464e89)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:14.094669 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a712a00-2b2a-4937-8469-dce20a226a6d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:14.256437 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-5a712a00-2b2a-4937-8469-dce20a226a6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '249024e12a8c41d3b831203719464e89', 'request_id': 'req-5a712a00-2b2a-4937-8469-dce20a226a6d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:14.272107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-5a712a00-2b2a-4937-8469-dce20a226a6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '249024e12a8c41d3b831203719464e89', 'request_id': 'req-5a712a00-2b2a-4937-8469-dce20a226a6d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:14.296032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-5a712a00-2b2a-4937-8469-dce20a226a6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '249024e12a8c41d3b831203719464e89', 'request_id': 'req-5a712a00-2b2a-4937-8469-dce20a226a6d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:14.335889 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-5a712a00-2b2a-4937-8469-dce20a226a6d admin admin] The token cache is being invalidate because project 249024e12a8c41d3b831203719464e89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:14.340286 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-5a712a00-2b2a-4937-8469-dce20a226a6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:14.343074 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-5a712a00-2b2a-4937-8469-dce20a226a6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:14.356027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 972/1960] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:14:13 2022] DELETE /identity/v3/projects/249024e12a8c41d3b831203719464e89 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:14.359128 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd1b34-e349-43b0-94b0-72f5628cfb8a None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:14.359722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd1b34-e349-43b0-94b0-72f5628cfb8a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:14.360054 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd1b34-e349-43b0-94b0-72f5628cfb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:14.566385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-93dd1b34-e349-43b0-94b0-72f5628cfb8a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:14.567933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-93dd1b34-e349-43b0-94b0-72f5628cfb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:14.713710 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:14.741322 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 989/1961] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:14.755173 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce4368-e4a2-4ec7-9504-70bf921091e5 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:14.755667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce4368-e4a2-4ec7-9504-70bf921091e5 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:14.756158 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce4368-e4a2-4ec7-9504-70bf921091e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:14.801722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-05ce4368-e4a2-4ec7-9504-70bf921091e5 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:14.806191 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-05ce4368-e4a2-4ec7-9504-70bf921091e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:14.825425 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 973/1962] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:14.834456 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd1b34-e349-43b0-94b0-72f5628cfb8a None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:14.859981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd1b34-e349-43b0-94b0-72f5628cfb8a None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:14.904089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cded2d-f0d2-4d01-80eb-afcc68a7ff18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:14.907761 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cded2d-f0d2-4d01-80eb-afcc68a7ff18 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:14.908262 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cded2d-f0d2-4d01-80eb-afcc68a7ff18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:14.913284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cded2d-f0d2-4d01-80eb-afcc68a7ff18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:14.915150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cded2d-f0d2-4d01-80eb-afcc68a7ff18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:14.915508 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cded2d-f0d2-4d01-80eb-afcc68a7ff18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:14.923180 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cded2d-f0d2-4d01-80eb-afcc68a7ff18 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:14.983351 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 990/1963] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:14.987787 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce4368-e4a2-4ec7-9504-70bf921091e5 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:15.000493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce4368-e4a2-4ec7-9504-70bf921091e5 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:15.019986 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f99b5-8494-422b-8c23-96c3a917537a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:15.022834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f99b5-8494-422b-8c23-96c3a917537a admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:15.022834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f99b5-8494-422b-8c23-96c3a917537a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:15.022834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f99b5-8494-422b-8c23-96c3a917537a admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:15.022834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f99b5-8494-422b-8c23-96c3a917537a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:15.031430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f99b5-8494-422b-8c23-96c3a917537a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:15.093306 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 974/1964] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:14 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:14:15.099808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cded2d-f0d2-4d01-80eb-afcc68a7ff18 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:15.178267 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cded2d-f0d2-4d01-80eb-afcc68a7ff18 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:15.361026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf4c6c-8a39-4219-b2f7-341121cfda2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:15.361846 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf4c6c-8a39-4219-b2f7-341121cfda2f admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:15.362286 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf4c6c-8a39-4219-b2f7-341121cfda2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:15.362659 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf4c6c-8a39-4219-b2f7-341121cfda2f admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:15.363529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf4c6c-8a39-4219-b2f7-341121cfda2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:15.365887 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf4c6c-8a39-4219-b2f7-341121cfda2f admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:15.406648 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 991/1965] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:15 2022] POST /identity/v3/users => generated 292 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:15.411820 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f99b5-8494-422b-8c23-96c3a917537a admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:15.448357 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f99b5-8494-422b-8c23-96c3a917537a admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:15.468914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b92754-017d-4eb5-8ac1-b636db3cc40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:15.476591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b92754-017d-4eb5-8ac1-b636db3cc40e admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:15.477405 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b92754-017d-4eb5-8ac1-b636db3cc40e admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:15.478032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b92754-017d-4eb5-8ac1-b636db3cc40e admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:15.479564 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b92754-017d-4eb5-8ac1-b636db3cc40e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:15.489835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b92754-017d-4eb5-8ac1-b636db3cc40e admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:15.521499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:15.536965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 975/1966] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:15.541840 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf4c6c-8a39-4219-b2f7-341121cfda2f admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:15.574210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf4c6c-8a39-4219-b2f7-341121cfda2f admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:15.597688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47500163-379f-4c99-a05a-f2123e3fe95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:15.598038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47500163-379f-4c99-a05a-f2123e3fe95d admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:15.598462 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47500163-379f-4c99-a05a-f2123e3fe95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:15.601288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47500163-379f-4c99-a05a-f2123e3fe95d admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:15.601288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47500163-379f-4c99-a05a-f2123e3fe95d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:15.606505 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47500163-379f-4c99-a05a-f2123e3fe95d admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:15.651657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 992/1967] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:15.657325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b92754-017d-4eb5-8ac1-b636db3cc40e admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:15.698491 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b92754-017d-4eb5-8ac1-b636db3cc40e admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:15.729479 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ef90a-6c8a-48d8-9aec-11ad1710fce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:15.731742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ef90a-6c8a-48d8-9aec-11ad1710fce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:15.731742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ef90a-6c8a-48d8-9aec-11ad1710fce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:15.731742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ef90a-6c8a-48d8-9aec-11ad1710fce9 admin admin] PATH_INFO: `/v3/projects/6e53c1f8874e41fc85575169493d427e/users/a20a8558378d489a8cb6b554a336a26c/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:15.790790 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ef90a-6c8a-48d8-9aec-11ad1710fce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e53c1f8874e41fc85575169493d427e, user_id=a20a8558378d489a8cb6b554a336a26c, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:15.798446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ef90a-6c8a-48d8-9aec-11ad1710fce9 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:15.849536 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 976/1968] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:15 2022] PUT /identity/v3/projects/6e53c1f8874e41fc85575169493d427e/users/a20a8558378d489a8cb6b554a336a26c/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:14:15.867608 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bcb48-0792-41f6-b88f-726848f3bb64 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:15.868566 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bcb48-0792-41f6-b88f-726848f3bb64 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:15.868975 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bcb48-0792-41f6-b88f-726848f3bb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:15.950437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-968bcb48-0792-41f6-b88f-726848f3bb64 None None] MFA Rules not processed for user `a20a8558378d489a8cb6b554a336a26c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:15.952492 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-968bcb48-0792-41f6-b88f-726848f3bb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:16.119990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 993/1969] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:16.125943 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ef90a-6c8a-48d8-9aec-11ad1710fce9 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:16.307808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ef90a-6c8a-48d8-9aec-11ad1710fce9 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:16.317595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d69b1a-c9b7-4a55-8d12-47c3f1f51af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:16.319057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d69b1a-c9b7-4a55-8d12-47c3f1f51af8 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:16.319605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d69b1a-c9b7-4a55-8d12-47c3f1f51af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:16.321662 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d69b1a-c9b7-4a55-8d12-47c3f1f51af8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:16.322716 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d69b1a-c9b7-4a55-8d12-47c3f1f51af8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:16.325213 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d69b1a-c9b7-4a55-8d12-47c3f1f51af8 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:16.400298 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 977/1970] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:16 2022] POST /identity/v3/projects => generated 383 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:16.401957 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bcb48-0792-41f6-b88f-726848f3bb64 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:16.455151 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bcb48-0792-41f6-b88f-726848f3bb64 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:16.769053 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:16.872783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317ac25-e443-428c-8148-466abf59ded7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:16.874421 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317ac25-e443-428c-8148-466abf59ded7 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:16.876543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317ac25-e443-428c-8148-466abf59ded7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:16.877420 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317ac25-e443-428c-8148-466abf59ded7 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:16.883462 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317ac25-e443-428c-8148-466abf59ded7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:16.888722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317ac25-e443-428c-8148-466abf59ded7 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:16.969640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d69b1a-c9b7-4a55-8d12-47c3f1f51af8 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:16.980583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 994/1971] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:16 2022] POST /identity/v3/users => generated 298 bytes in 566 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:17.015491 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d69b1a-c9b7-4a55-8d12-47c3f1f51af8 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:17.117180 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20a348-e340-4c57-be64-b72ee73d8a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:17.134942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20a348-e340-4c57-be64-b72ee73d8a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:17.136342 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20a348-e340-4c57-be64-b72ee73d8a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:17.138520 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20a348-e340-4c57-be64-b72ee73d8a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:17.143119 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20a348-e340-4c57-be64-b72ee73d8a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:17.147110 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20a348-e340-4c57-be64-b72ee73d8a59 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:17.204771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317ac25-e443-428c-8148-466abf59ded7 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:17.206928 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 978/1972] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:17.247384 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317ac25-e443-428c-8148-466abf59ded7 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:17.282370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aac1e5-19e8-4949-b261-b7a048e6091a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:17.283103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aac1e5-19e8-4949-b261-b7a048e6091a admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:17.283491 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aac1e5-19e8-4949-b261-b7a048e6091a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:17.284583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aac1e5-19e8-4949-b261-b7a048e6091a admin admin] PATH_INFO: `/v3/projects/2a6ac2ef555f416ead3e42817ecdead5/users/d43257480fb9488b829d34ccae6bd56b/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:17.347141 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aac1e5-19e8-4949-b261-b7a048e6091a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a6ac2ef555f416ead3e42817ecdead5, user_id=d43257480fb9488b829d34ccae6bd56b, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:17.357699 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aac1e5-19e8-4949-b261-b7a048e6091a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:17.431404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20a348-e340-4c57-be64-b72ee73d8a59 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:17.442403 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 995/1973] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:17 2022] PUT /identity/v3/projects/2a6ac2ef555f416ead3e42817ecdead5/users/d43257480fb9488b829d34ccae6bd56b/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:14:17.473624 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20a348-e340-4c57-be64-b72ee73d8a59 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:17.719822 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a9589-fd8b-4600-8419-8e4d1aa041ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:17.720943 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a9589-fd8b-4600-8419-8e4d1aa041ae admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:17.721466 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a9589-fd8b-4600-8419-8e4d1aa041ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:17.731803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a9589-fd8b-4600-8419-8e4d1aa041ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:17.737195 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a9589-fd8b-4600-8419-8e4d1aa041ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:17.752244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a9589-fd8b-4600-8419-8e4d1aa041ae admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:17.799627 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 979/1974] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:17.808012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aac1e5-19e8-4949-b261-b7a048e6091a admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:17.875961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aac1e5-19e8-4949-b261-b7a048e6091a admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:17.909300 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783b134-fc57-4e28-ac2e-29568271ff2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:17.911069 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783b134-fc57-4e28-ac2e-29568271ff2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:17.914360 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783b134-fc57-4e28-ac2e-29568271ff2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:17.916442 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783b134-fc57-4e28-ac2e-29568271ff2f admin admin] PATH_INFO: `/v3/projects/2a6ac2ef555f416ead3e42817ecdead5/users/d43257480fb9488b829d34ccae6bd56b/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:17.927267 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783b134-fc57-4e28-ac2e-29568271ff2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a6ac2ef555f416ead3e42817ecdead5, user_id=d43257480fb9488b829d34ccae6bd56b, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:17.938113 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783b134-fc57-4e28-ac2e-29568271ff2f admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:17.997705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 996/1975] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:17 2022] PUT /identity/v3/projects/2a6ac2ef555f416ead3e42817ecdead5/users/d43257480fb9488b829d34ccae6bd56b/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:18.000207 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06987ac-33e0-4298-834f-bcac936f5f48 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:18.000775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06987ac-33e0-4298-834f-bcac936f5f48 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:18.001109 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06987ac-33e0-4298-834f-bcac936f5f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:18.085009 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-f06987ac-33e0-4298-834f-bcac936f5f48 None None] MFA Rules not processed for user `d43257480fb9488b829d34ccae6bd56b`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:18.097054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-f06987ac-33e0-4298-834f-bcac936f5f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:18.217005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:18.330107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 980/1976] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:18.354125 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783b134-fc57-4e28-ac2e-29568271ff2f admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:18.357614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0783b134-fc57-4e28-ac2e-29568271ff2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:18.582325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783b134-fc57-4e28-ac2e-29568271ff2f admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:18.587002 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60887dd-461b-4394-bd9b-55ab6f32fdb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:18.587793 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60887dd-461b-4394-bd9b-55ab6f32fdb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:18.587982 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60887dd-461b-4394-bd9b-55ab6f32fdb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:18.588265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60887dd-461b-4394-bd9b-55ab6f32fdb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:18.601482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60887dd-461b-4394-bd9b-55ab6f32fdb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:18.603659 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60887dd-461b-4394-bd9b-55ab6f32fdb0 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:18.617387 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 997/1977] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:18.854808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06987ac-33e0-4298-834f-bcac936f5f48 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:18.880562 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06987ac-33e0-4298-834f-bcac936f5f48 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:18.890278 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cffbd11-96cf-444e-8a10-0de7ace52478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:18.891807 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffbd11-96cf-444e-8a10-0de7ace52478 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:18.892616 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffbd11-96cf-444e-8a10-0de7ace52478 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:18.893307 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffbd11-96cf-444e-8a10-0de7ace52478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:19.236931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cffbd11-96cf-444e-8a10-0de7ace52478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:19.240001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cffbd11-96cf-444e-8a10-0de7ace52478 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:19.271265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 981/1978] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:19.648137 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23c9be-5e28-4873-93e5-e9c2370cdb87 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:19.649592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23c9be-5e28-4873-93e5-e9c2370cdb87 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:19.649592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23c9be-5e28-4873-93e5-e9c2370cdb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:19.726465 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-6d23c9be-5e28-4873-93e5-e9c2370cdb87 None None] MFA Rules not processed for user `16f05877e10442d2a8938f6805eff842`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:19.731008 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-6d23c9be-5e28-4873-93e5-e9c2370cdb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:19.824832 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cffbd11-96cf-444e-8a10-0de7ace52478 service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:19.829191 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-3cffbd11-96cf-444e-8a10-0de7ace52478 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:19.886088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cffbd11-96cf-444e-8a10-0de7ace52478 service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:19.906345 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18632bc-1f4e-4fc3-ad29-87ff63432489 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:19.908780 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18632bc-1f4e-4fc3-ad29-87ff63432489 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:19.910404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18632bc-1f4e-4fc3-ad29-87ff63432489 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:19.910957 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18632bc-1f4e-4fc3-ad29-87ff63432489 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:19.919123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b18632bc-1f4e-4fc3-ad29-87ff63432489 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:19.971748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 998/1979] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:19.981753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23c9be-5e28-4873-93e5-e9c2370cdb87 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:19.991413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-6d23c9be-5e28-4873-93e5-e9c2370cdb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:20.132830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:20.217768 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18632bc-1f4e-4fc3-ad29-87ff63432489 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:20.219026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18632bc-1f4e-4fc3-ad29-87ff63432489 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:20.259108 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23c9be-5e28-4873-93e5-e9c2370cdb87 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:20.282250 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfccdd3-9f9d-4f88-8179-471814877916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:20.282773 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 982/1980] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:20.283619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfccdd3-9f9d-4f88-8179-471814877916 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:20.284132 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfccdd3-9f9d-4f88-8179-471814877916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:20.284674 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfccdd3-9f9d-4f88-8179-471814877916 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:20.286077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfccdd3-9f9d-4f88-8179-471814877916 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:20.292070 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfccdd3-9f9d-4f88-8179-471814877916 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:20.409269 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 999/1981] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:19 2022] POST /identity/v3/projects => generated 383 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:20.411170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18632bc-1f4e-4fc3-ad29-87ff63432489 service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:20.449904 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18632bc-1f4e-4fc3-ad29-87ff63432489 service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:20.654362 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d36fa-8341-40f6-a87e-af2f27b13bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:20.658148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d36fa-8341-40f6-a87e-af2f27b13bab admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:20.658677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d36fa-8341-40f6-a87e-af2f27b13bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:20.659254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d36fa-8341-40f6-a87e-af2f27b13bab admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:20.664725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d36fa-8341-40f6-a87e-af2f27b13bab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:20.669771 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d36fa-8341-40f6-a87e-af2f27b13bab admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:20.691577 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:20.772694 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 983/1982] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:20 2022] POST /identity/v3/users => generated 298 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:20.779218 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfccdd3-9f9d-4f88-8179-471814877916 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:20.822606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfccdd3-9f9d-4f88-8179-471814877916 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:20.833661 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfed6e3-4f1e-4874-bd1f-eb9a77176175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:20.834736 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfed6e3-4f1e-4874-bd1f-eb9a77176175 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:20.835383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfed6e3-4f1e-4874-bd1f-eb9a77176175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:20.836266 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfed6e3-4f1e-4874-bd1f-eb9a77176175 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:20.838329 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfed6e3-4f1e-4874-bd1f-eb9a77176175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:20.840146 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfed6e3-4f1e-4874-bd1f-eb9a77176175 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:20.855311 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1000/1983] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:20.861557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d36fa-8341-40f6-a87e-af2f27b13bab admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:20.890806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d36fa-8341-40f6-a87e-af2f27b13bab admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:20.931835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dcaeeb-98d1-40ba-8d6e-5d8d2d8af224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:20.934143 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dcaeeb-98d1-40ba-8d6e-5d8d2d8af224 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:20.934662 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dcaeeb-98d1-40ba-8d6e-5d8d2d8af224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:20.935263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dcaeeb-98d1-40ba-8d6e-5d8d2d8af224 admin admin] PATH_INFO: `/v3/projects/0cae23fd696d4f579f5c8e6e6d68ce45/users/9945e4ffbc2548d5962d581aa905bb5f/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:21.023478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dcaeeb-98d1-40ba-8d6e-5d8d2d8af224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cae23fd696d4f579f5c8e6e6d68ce45, user_id=9945e4ffbc2548d5962d581aa905bb5f, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:21.037877 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dcaeeb-98d1-40ba-8d6e-5d8d2d8af224 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:21.094180 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 984/1984] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:20 2022] PUT /identity/v3/projects/0cae23fd696d4f579f5c8e6e6d68ce45/users/9945e4ffbc2548d5962d581aa905bb5f/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:21.100947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfed6e3-4f1e-4874-bd1f-eb9a77176175 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:21.152043 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfed6e3-4f1e-4874-bd1f-eb9a77176175 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:21.351656 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a1824-560f-4ae4-b3f2-420dbab230ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:21.353522 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a1824-560f-4ae4-b3f2-420dbab230ed admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:21.354029 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a1824-560f-4ae4-b3f2-420dbab230ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:21.354540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a1824-560f-4ae4-b3f2-420dbab230ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:21.362524 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a1824-560f-4ae4-b3f2-420dbab230ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:21.362524 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a1824-560f-4ae4-b3f2-420dbab230ed admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:21.386175 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1001/1985] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:21.391427 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dcaeeb-98d1-40ba-8d6e-5d8d2d8af224 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:21.417977 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dcaeeb-98d1-40ba-8d6e-5d8d2d8af224 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:21.434906 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbbba2-3100-4136-b97e-7a86f515b12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:21.438127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbbba2-3100-4136-b97e-7a86f515b12d admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:21.440261 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbbba2-3100-4136-b97e-7a86f515b12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:21.441312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbbba2-3100-4136-b97e-7a86f515b12d admin admin] PATH_INFO: `/v3/projects/0cae23fd696d4f579f5c8e6e6d68ce45/users/9945e4ffbc2548d5962d581aa905bb5f/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:21.447788 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bbbba2-3100-4136-b97e-7a86f515b12d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cae23fd696d4f579f5c8e6e6d68ce45, user_id=9945e4ffbc2548d5962d581aa905bb5f, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:21.454185 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bbbba2-3100-4136-b97e-7a86f515b12d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:21.488041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 985/1986] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:21 2022] PUT /identity/v3/projects/0cae23fd696d4f579f5c8e6e6d68ce45/users/9945e4ffbc2548d5962d581aa905bb5f/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:23.428513 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a1824-560f-4ae4-b3f2-420dbab230ed admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.453202 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a1824-560f-4ae4-b3f2-420dbab230ed admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.595322 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16302c44-5a42-4e66-a165-efb5e1b3cfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.596158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16302c44-5a42-4e66-a165-efb5e1b3cfd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.596656 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16302c44-5a42-4e66-a165-efb5e1b3cfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.597099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16302c44-5a42-4e66-a165-efb5e1b3cfd7 admin admin] PATH_INFO: `/v3/users/a20a8558378d489a8cb6b554a336a26c` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.600709 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16302c44-5a42-4e66-a165-efb5e1b3cfd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a20a8558378d489a8cb6b554a336a26c)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.608556 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16302c44-5a42-4e66-a165-efb5e1b3cfd7 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.721992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:23.725356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-16302c44-5a42-4e66-a165-efb5e1b3cfd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a20a8558378d489a8cb6b554a336a26c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:23.740148 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-16302c44-5a42-4e66-a165-efb5e1b3cfd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a20a8558378d489a8cb6b554a336a26c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:23.761680 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-16302c44-5a42-4e66-a165-efb5e1b3cfd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a20a8558378d489a8cb6b554a336a26c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:23.762777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-16302c44-5a42-4e66-a165-efb5e1b3cfd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a20a8558378d489a8cb6b554a336a26c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:23.763744 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-16302c44-5a42-4e66-a165-efb5e1b3cfd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a20a8558378d489a8cb6b554a336a26c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:23.818561 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1002/1987] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:14:23 2022] DELETE /identity/v3/users/a20a8558378d489a8cb6b554a336a26c => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:23.821698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b568c-2e08-464f-a443-1c3905f30d58 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.822520 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b568c-2e08-464f-a443-1c3905f30d58 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.823051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b568c-2e08-464f-a443-1c3905f30d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.880145 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-580b568c-2e08-464f-a443-1c3905f30d58 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:23.882953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-580b568c-2e08-464f-a443-1c3905f30d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:24.023258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 986/1988] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:24.039057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16302c44-5a42-4e66-a165-efb5e1b3cfd7 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:24.044555 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-16302c44-5a42-4e66-a165-efb5e1b3cfd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:24.283052 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16302c44-5a42-4e66-a165-efb5e1b3cfd7 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:24.298486 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550c0e4-503c-4ad5-aed0-65e85a733561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:24.299828 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550c0e4-503c-4ad5-aed0-65e85a733561 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.300209 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550c0e4-503c-4ad5-aed0-65e85a733561 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.300742 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550c0e4-503c-4ad5-aed0-65e85a733561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:24.333750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550c0e4-503c-4ad5-aed0-65e85a733561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:24.342245 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550c0e4-503c-4ad5-aed0-65e85a733561 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:24.442570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1003/1989] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:25.484932 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b568c-2e08-464f-a443-1c3905f30d58 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:25.489822 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-580b568c-2e08-464f-a443-1c3905f30d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:25.560408 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b568c-2e08-464f-a443-1c3905f30d58 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:25.589666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fba91-f40c-4aa4-8dbd-038f0582790b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:25.610562 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fba91-f40c-4aa4-8dbd-038f0582790b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:25.611546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fba91-f40c-4aa4-8dbd-038f0582790b admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:25.611989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fba91-f40c-4aa4-8dbd-038f0582790b admin admin] PATH_INFO: `/v3/projects/6e53c1f8874e41fc85575169493d427e` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:25.622389 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fba91-f40c-4aa4-8dbd-038f0582790b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e53c1f8874e41fc85575169493d427e)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:25.626691 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fba91-f40c-4aa4-8dbd-038f0582790b admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:25.868648 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e15fba91-f40c-4aa4-8dbd-038f0582790b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e53c1f8874e41fc85575169493d427e', 'request_id': 'req-e15fba91-f40c-4aa4-8dbd-038f0582790b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:25.910643 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e15fba91-f40c-4aa4-8dbd-038f0582790b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e53c1f8874e41fc85575169493d427e', 'request_id': 'req-e15fba91-f40c-4aa4-8dbd-038f0582790b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:25.927584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e15fba91-f40c-4aa4-8dbd-038f0582790b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e53c1f8874e41fc85575169493d427e', 'request_id': 'req-e15fba91-f40c-4aa4-8dbd-038f0582790b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:26.019508 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e15fba91-f40c-4aa4-8dbd-038f0582790b admin admin] The token cache is being invalidate because project 6e53c1f8874e41fc85575169493d427e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:26.020387 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e15fba91-f40c-4aa4-8dbd-038f0582790b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:26.028496 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e15fba91-f40c-4aa4-8dbd-038f0582790b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:26.032819 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 987/1990] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:14:25 2022] DELETE /identity/v3/projects/6e53c1f8874e41fc85575169493d427e => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:26.036805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550c0e4-503c-4ad5-aed0-65e85a733561 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:26.054787 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550c0e4-503c-4ad5-aed0-65e85a733561 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:26.162558 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabe229-8fbe-49e8-bcc8-b1b10c080292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:26.163558 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabe229-8fbe-49e8-bcc8-b1b10c080292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:26.165883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabe229-8fbe-49e8-bcc8-b1b10c080292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:26.166860 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabe229-8fbe-49e8-bcc8-b1b10c080292 admin admin] PATH_INFO: `/v3/users/d43257480fb9488b829d34ccae6bd56b` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:26.169841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabe229-8fbe-49e8-bcc8-b1b10c080292 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d43257480fb9488b829d34ccae6bd56b)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:26.177736 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabe229-8fbe-49e8-bcc8-b1b10c080292 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:26.316352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4eabe229-8fbe-49e8-bcc8-b1b10c080292 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd43257480fb9488b829d34ccae6bd56b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:26.325702 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4eabe229-8fbe-49e8-bcc8-b1b10c080292 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd43257480fb9488b829d34ccae6bd56b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:26.341435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4eabe229-8fbe-49e8-bcc8-b1b10c080292 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd43257480fb9488b829d34ccae6bd56b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:26.344168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4eabe229-8fbe-49e8-bcc8-b1b10c080292 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd43257480fb9488b829d34ccae6bd56b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:26.355317 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4eabe229-8fbe-49e8-bcc8-b1b10c080292 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd43257480fb9488b829d34ccae6bd56b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:26.367892 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:26.409074 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1004/1991] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:14:26 2022] DELETE /identity/v3/users/d43257480fb9488b829d34ccae6bd56b => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:27.296660 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fba91-f40c-4aa4-8dbd-038f0582790b admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:27.300515 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e15fba91-f40c-4aa4-8dbd-038f0582790b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:27.338352 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fba91-f40c-4aa4-8dbd-038f0582790b admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:27.368220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabe229-8fbe-49e8-bcc8-b1b10c080292 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:27.369902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4eabe229-8fbe-49e8-bcc8-b1b10c080292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:27.522403 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:27.583106 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52348a-eb24-4460-9377-f9210e69f342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:27.584093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52348a-eb24-4460-9377-f9210e69f342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:27.585170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52348a-eb24-4460-9377-f9210e69f342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:27.587009 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52348a-eb24-4460-9377-f9210e69f342 admin admin] PATH_INFO: `/v3/users/e3455625067a4924a2095f89a129d454` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:27.593248 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52348a-eb24-4460-9377-f9210e69f342 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3455625067a4924a2095f89a129d454)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:27.598481 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52348a-eb24-4460-9377-f9210e69f342 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:27.612838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabe229-8fbe-49e8-bcc8-b1b10c080292 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:27.624139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737178ff-163a-45ed-8b1c-63671b9f8e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:27.625663 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737178ff-163a-45ed-8b1c-63671b9f8e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:27.626250 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737178ff-163a-45ed-8b1c-63671b9f8e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:27.626750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737178ff-163a-45ed-8b1c-63671b9f8e4d admin admin] PATH_INFO: `/v3/projects/2a6ac2ef555f416ead3e42817ecdead5` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:27.629408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737178ff-163a-45ed-8b1c-63671b9f8e4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a6ac2ef555f416ead3e42817ecdead5)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:27.635223 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737178ff-163a-45ed-8b1c-63671b9f8e4d admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:27.737616 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-737178ff-163a-45ed-8b1c-63671b9f8e4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a6ac2ef555f416ead3e42817ecdead5', 'request_id': 'req-737178ff-163a-45ed-8b1c-63671b9f8e4d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:27.772856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-737178ff-163a-45ed-8b1c-63671b9f8e4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a6ac2ef555f416ead3e42817ecdead5', 'request_id': 'req-737178ff-163a-45ed-8b1c-63671b9f8e4d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:27.787454 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-ae52348a-eb24-4460-9377-f9210e69f342 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3455625067a4924a2095f89a129d454'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:27.790390 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-ae52348a-eb24-4460-9377-f9210e69f342 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3455625067a4924a2095f89a129d454'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:27.792658 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-737178ff-163a-45ed-8b1c-63671b9f8e4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a6ac2ef555f416ead3e42817ecdead5', 'request_id': 'req-737178ff-163a-45ed-8b1c-63671b9f8e4d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:27.811814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-737178ff-163a-45ed-8b1c-63671b9f8e4d admin admin] The token cache is being invalidate because project 2a6ac2ef555f416ead3e42817ecdead5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:27.812652 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-737178ff-163a-45ed-8b1c-63671b9f8e4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:27.814181 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-737178ff-163a-45ed-8b1c-63671b9f8e4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:27.817393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1005/1992] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:14:27 2022] DELETE /identity/v3/projects/2a6ac2ef555f416ead3e42817ecdead5 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:27.825006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-ae52348a-eb24-4460-9377-f9210e69f342 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3455625067a4924a2095f89a129d454'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:27.826084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-ae52348a-eb24-4460-9377-f9210e69f342 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3455625067a4924a2095f89a129d454'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:27.828237 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98220b6d-8690-410e-8853-97b76d22a00f None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:27.829841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98220b6d-8690-410e-8853-97b76d22a00f None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:27.830379 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98220b6d-8690-410e-8853-97b76d22a00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:27.865074 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-ae52348a-eb24-4460-9377-f9210e69f342 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3455625067a4924a2095f89a129d454'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:27.895051 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-98220b6d-8690-410e-8853-97b76d22a00f None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:27.898716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-98220b6d-8690-410e-8853-97b76d22a00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:27.901721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 988/1993] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:14:27 2022] DELETE /identity/v3/users/e3455625067a4924a2095f89a129d454 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:27.908001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871802fc-bfbc-4e16-80ed-46662b56b865 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:27.908411 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871802fc-bfbc-4e16-80ed-46662b56b865 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:27.908828 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871802fc-bfbc-4e16-80ed-46662b56b865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:27.986967 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-871802fc-bfbc-4e16-80ed-46662b56b865 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:27.991815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-871802fc-bfbc-4e16-80ed-46662b56b865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:28.198300 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1006/1994] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:28.212788 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d4385-4da3-4126-89df-9b3e1be3818f None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:28.213220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d4385-4da3-4126-89df-9b3e1be3818f None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:28.213623 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d4385-4da3-4126-89df-9b3e1be3818f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:28.268731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-bb6d4385-4da3-4126-89df-9b3e1be3818f None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:28.270662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-bb6d4385-4da3-4126-89df-9b3e1be3818f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:28.271545 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 989/1995] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:28.286043 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871802fc-bfbc-4e16-80ed-46662b56b865 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:28.291460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-871802fc-bfbc-4e16-80ed-46662b56b865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:28.312285 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1007/1996] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:28.316393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d4385-4da3-4126-89df-9b3e1be3818f None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:28.340158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d4385-4da3-4126-89df-9b3e1be3818f None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:28.358953 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb49ec0-d480-4553-a769-14fe9f48cf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:28.359784 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb49ec0-d480-4553-a769-14fe9f48cf45 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:28.359924 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb49ec0-d480-4553-a769-14fe9f48cf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:28.360142 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb49ec0-d480-4553-a769-14fe9f48cf45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:28.361705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb49ec0-d480-4553-a769-14fe9f48cf45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:28.362095 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb49ec0-d480-4553-a769-14fe9f48cf45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:28.365263 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb49ec0-d480-4553-a769-14fe9f48cf45 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:28.369864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:28.388339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1008/1997] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:14:28.392455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb49ec0-d480-4553-a769-14fe9f48cf45 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:28.413830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb49ec0-d480-4553-a769-14fe9f48cf45 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:28.433703 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befa79e-2c76-4f85-a78d-652bda1ae9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:28.434339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befa79e-2c76-4f85-a78d-652bda1ae9a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:28.434630 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befa79e-2c76-4f85-a78d-652bda1ae9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:28.434908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befa79e-2c76-4f85-a78d-652bda1ae9a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:28.435611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9befa79e-2c76-4f85-a78d-652bda1ae9a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:28.437763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9befa79e-2c76-4f85-a78d-652bda1ae9a9 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:28.461659 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871802fc-bfbc-4e16-80ed-46662b56b865 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:28.485629 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acf855-7d12-45c2-b0c9-c0e7a41fa1fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:28.487613 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acf855-7d12-45c2-b0c9-c0e7a41fa1fe service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:28.495582 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acf855-7d12-45c2-b0c9-c0e7a41fa1fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:28.496137 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acf855-7d12-45c2-b0c9-c0e7a41fa1fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:28.518483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1009/1998] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:28 2022] POST /identity/v3/projects => generated 407 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:14:28.525759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befa79e-2c76-4f85-a78d-652bda1ae9a9 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:28.529683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96acf855-7d12-45c2-b0c9-c0e7a41fa1fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:28.533092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96acf855-7d12-45c2-b0c9-c0e7a41fa1fe service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:28.568234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 990/1999] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:28.678384 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befa79e-2c76-4f85-a78d-652bda1ae9a9 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:28.687756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fbd93-5f59-4bc1-9963-de0d7a928817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:28.689002 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fbd93-5f59-4bc1-9963-de0d7a928817 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:28.690575 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fbd93-5f59-4bc1-9963-de0d7a928817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:28.691123 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fbd93-5f59-4bc1-9963-de0d7a928817 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:28.692557 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60fbd93-5f59-4bc1-9963-de0d7a928817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:28.697224 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60fbd93-5f59-4bc1-9963-de0d7a928817 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:28.790854 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1010/2000] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:28 2022] POST /identity/v3/users => generated 304 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:14:28.796282 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acf855-7d12-45c2-b0c9-c0e7a41fa1fe service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:28.816864 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:28.842659 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acf855-7d12-45c2-b0c9-c0e7a41fa1fe service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:28.859381 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b25b72-0404-4f33-8dae-b5dc6916dfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:28.860078 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b25b72-0404-4f33-8dae-b5dc6916dfdb admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:28.860364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b25b72-0404-4f33-8dae-b5dc6916dfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:28.860666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b25b72-0404-4f33-8dae-b5dc6916dfdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:28.862438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b25b72-0404-4f33-8dae-b5dc6916dfdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:28.866275 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b25b72-0404-4f33-8dae-b5dc6916dfdb admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:28.899160 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 991/2001] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:28.899812 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fbd93-5f59-4bc1-9963-de0d7a928817 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:28.919630 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fbd93-5f59-4bc1-9963-de0d7a928817 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:28.932146 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49296abc-084d-4113-9617-780a14fba107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:28.933637 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49296abc-084d-4113-9617-780a14fba107 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:28.934008 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49296abc-084d-4113-9617-780a14fba107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:28.934353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49296abc-084d-4113-9617-780a14fba107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:28.935289 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49296abc-084d-4113-9617-780a14fba107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:28.937299 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49296abc-084d-4113-9617-780a14fba107 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:28.959666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b25b72-0404-4f33-8dae-b5dc6916dfdb admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:28.965609 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1011/2002] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:14:28.974639 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b25b72-0404-4f33-8dae-b5dc6916dfdb admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:28.982583 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ff4d0-0405-4c9d-8d67-f6d9d1cd369f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:28.983710 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ff4d0-0405-4c9d-8d67-f6d9d1cd369f admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:28.985146 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ff4d0-0405-4c9d-8d67-f6d9d1cd369f admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:28.985636 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ff4d0-0405-4c9d-8d67-f6d9d1cd369f admin admin] PATH_INFO: `/v3/projects/49d083b0c8514ab2b458e0e8ad485bee/users/2d67bc3050ce4fc2bfab6d2f8e578604/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:29.011376 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ff4d0-0405-4c9d-8d67-f6d9d1cd369f admin admin] RBAC: Authorizing `identity:create_grant(project_id=49d083b0c8514ab2b458e0e8ad485bee, user_id=2d67bc3050ce4fc2bfab6d2f8e578604, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:29.019242 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ff4d0-0405-4c9d-8d67-f6d9d1cd369f admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:29.105219 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 992/2003] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:28 2022] PUT /identity/v3/projects/49d083b0c8514ab2b458e0e8ad485bee/users/2d67bc3050ce4fc2bfab6d2f8e578604/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:29.106768 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a257d8-e6c4-4df4-9192-835143a67460 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:29.107612 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a257d8-e6c4-4df4-9192-835143a67460 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:29.108907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a257d8-e6c4-4df4-9192-835143a67460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:29.304212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-e9a257d8-e6c4-4df4-9192-835143a67460 None None] MFA Rules not processed for user `2d67bc3050ce4fc2bfab6d2f8e578604`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:29.307195 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e9a257d8-e6c4-4df4-9192-835143a67460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:29.589695 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ff4d0-0405-4c9d-8d67-f6d9d1cd369f admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:29.594138 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ae6ff4d0-0405-4c9d-8d67-f6d9d1cd369f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:29.610327 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1012/2004] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:29 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:29.612590 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a257d8-e6c4-4df4-9192-835143a67460 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:29.719289 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:29.768164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a257d8-e6c4-4df4-9192-835143a67460 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:29.784682 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb209f51-f02f-4e7f-8faa-f9e184cc0a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:29.793943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb209f51-f02f-4e7f-8faa-f9e184cc0a9d admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:29.795601 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb209f51-f02f-4e7f-8faa-f9e184cc0a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:29.799834 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb209f51-f02f-4e7f-8faa-f9e184cc0a9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:29.802349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb209f51-f02f-4e7f-8faa-f9e184cc0a9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:29.805550 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ff4d0-0405-4c9d-8d67-f6d9d1cd369f admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:29.811340 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb209f51-f02f-4e7f-8faa-f9e184cc0a9d admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:29.813605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09b909-b570-4cac-85ae-2c40eca9d691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:29.814902 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09b909-b570-4cac-85ae-2c40eca9d691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:29.815448 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09b909-b570-4cac-85ae-2c40eca9d691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:29.816075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09b909-b570-4cac-85ae-2c40eca9d691 admin admin] PATH_INFO: `/v3/projects/652cef8527db4764ad1eb4fbf60704b6` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:29.820269 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09b909-b570-4cac-85ae-2c40eca9d691 admin admin] RBAC: Authorizing `identity:delete_project(project_id=652cef8527db4764ad1eb4fbf60704b6)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:29.826404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09b909-b570-4cac-85ae-2c40eca9d691 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:29.895055 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1013/2005] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:29 2022] POST /identity/v3/projects => generated 409 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:29.900056 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb209f51-f02f-4e7f-8faa-f9e184cc0a9d admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:29.938956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6a09b909-b570-4cac-85ae-2c40eca9d691 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '652cef8527db4764ad1eb4fbf60704b6', 'request_id': 'req-6a09b909-b570-4cac-85ae-2c40eca9d691'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:29.959211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6a09b909-b570-4cac-85ae-2c40eca9d691 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '652cef8527db4764ad1eb4fbf60704b6', 'request_id': 'req-6a09b909-b570-4cac-85ae-2c40eca9d691'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:29.968578 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6a09b909-b570-4cac-85ae-2c40eca9d691 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '652cef8527db4764ad1eb4fbf60704b6', 'request_id': 'req-6a09b909-b570-4cac-85ae-2c40eca9d691'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:30.015528 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6a09b909-b570-4cac-85ae-2c40eca9d691 admin admin] The token cache is being invalidate because project 652cef8527db4764ad1eb4fbf60704b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:30.015800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6a09b909-b570-4cac-85ae-2c40eca9d691 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:30.017585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6a09b909-b570-4cac-85ae-2c40eca9d691 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:30.025584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 993/2006] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:14:29 2022] DELETE /identity/v3/projects/652cef8527db4764ad1eb4fbf60704b6 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:30.026647 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09b909-b570-4cac-85ae-2c40eca9d691 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:30.029785 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-6a09b909-b570-4cac-85ae-2c40eca9d691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:30.056702 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb209f51-f02f-4e7f-8faa-f9e184cc0a9d admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:30.218403 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:30.230770 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09b909-b570-4cac-85ae-2c40eca9d691 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:30.249240 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e14a1-bfa8-4e81-93b3-8039d2b9362c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:30.249960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e14a1-bfa8-4e81-93b3-8039d2b9362c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:30.250376 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e14a1-bfa8-4e81-93b3-8039d2b9362c admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:30.250577 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e14a1-bfa8-4e81-93b3-8039d2b9362c admin admin] PATH_INFO: `/v3/users/9f4b2ffdc8b44d8fbd15d967d9078a77` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:30.252730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e14a1-bfa8-4e81-93b3-8039d2b9362c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f4b2ffdc8b44d8fbd15d967d9078a77)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:30.255416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e14a1-bfa8-4e81-93b3-8039d2b9362c admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:30.267691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff51c4-45b4-4803-bb53-bfa23d0f18cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:30.268344 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff51c4-45b4-4803-bb53-bfa23d0f18cf admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:30.268699 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff51c4-45b4-4803-bb53-bfa23d0f18cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:30.269094 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff51c4-45b4-4803-bb53-bfa23d0f18cf admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:30.269943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff51c4-45b4-4803-bb53-bfa23d0f18cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:30.273008 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff51c4-45b4-4803-bb53-bfa23d0f18cf admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:30.315997 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1014/2007] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:29 2022] POST /identity/v3/users => generated 311 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:30.317842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff51c4-45b4-4803-bb53-bfa23d0f18cf admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:30.321203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-39ff51c4-45b4-4803-bb53-bfa23d0f18cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:30.342176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff51c4-45b4-4803-bb53-bfa23d0f18cf admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:30.352786 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd5ebd-9120-4f97-81d6-b9f3bf0244d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:30.354492 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd5ebd-9120-4f97-81d6-b9f3bf0244d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:30.354492 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd5ebd-9120-4f97-81d6-b9f3bf0244d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:30.354806 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd5ebd-9120-4f97-81d6-b9f3bf0244d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:30.355739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd5ebd-9120-4f97-81d6-b9f3bf0244d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:30.359167 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd5ebd-9120-4f97-81d6-b9f3bf0244d4 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:30.383046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1015/2008] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:30.389091 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd5ebd-9120-4f97-81d6-b9f3bf0244d4 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:30.390815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-d44e14a1-bfa8-4e81-93b3-8039d2b9362c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f4b2ffdc8b44d8fbd15d967d9078a77'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:30.391618 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-d44e14a1-bfa8-4e81-93b3-8039d2b9362c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f4b2ffdc8b44d8fbd15d967d9078a77'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:30.415540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-d44e14a1-bfa8-4e81-93b3-8039d2b9362c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f4b2ffdc8b44d8fbd15d967d9078a77'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:30.416418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-d44e14a1-bfa8-4e81-93b3-8039d2b9362c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f4b2ffdc8b44d8fbd15d967d9078a77'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:30.418256 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd5ebd-9120-4f97-81d6-b9f3bf0244d4 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:30.432939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e262d-bed7-4ef5-8e7b-25c1032bcc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:30.442664 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e262d-bed7-4ef5-8e7b-25c1032bcc77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:30.443347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e262d-bed7-4ef5-8e7b-25c1032bcc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:30.443841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e262d-bed7-4ef5-8e7b-25c1032bcc77 admin admin] PATH_INFO: `/v3/projects/bbf4b01c3cdd4b8f98406b488682f4aa/users/a2246d3375eb4581b0e9c0d2204d725d/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:30.450407 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-d44e14a1-bfa8-4e81-93b3-8039d2b9362c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f4b2ffdc8b44d8fbd15d967d9078a77'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:30.486334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 994/2009] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:14:30 2022] DELETE /identity/v3/users/9f4b2ffdc8b44d8fbd15d967d9078a77 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:30.504945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e262d-bed7-4ef5-8e7b-25c1032bcc77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbf4b01c3cdd4b8f98406b488682f4aa, user_id=a2246d3375eb4581b0e9c0d2204d725d, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:30.509519 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e262d-bed7-4ef5-8e7b-25c1032bcc77 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:30.558063 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1016/2010] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:30 2022] PUT /identity/v3/projects/bbf4b01c3cdd4b8f98406b488682f4aa/users/a2246d3375eb4581b0e9c0d2204d725d/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:30.563383 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e14a1-bfa8-4e81-93b3-8039d2b9362c admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:30.565259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d44e14a1-bfa8-4e81-93b3-8039d2b9362c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:30.758604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e14a1-bfa8-4e81-93b3-8039d2b9362c admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:30.941671 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c313ba-89fd-4102-97c9-efbfacdd2edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:30.942809 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c313ba-89fd-4102-97c9-efbfacdd2edb admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:30.943332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c313ba-89fd-4102-97c9-efbfacdd2edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:30.943998 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c313ba-89fd-4102-97c9-efbfacdd2edb admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:30.945595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c313ba-89fd-4102-97c9-efbfacdd2edb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:30.950001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c313ba-89fd-4102-97c9-efbfacdd2edb admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:30.972619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 995/2011] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:30.978404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e262d-bed7-4ef5-8e7b-25c1032bcc77 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:31.010529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e262d-bed7-4ef5-8e7b-25c1032bcc77 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:31.021286 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b93e08-0c1b-4b86-8765-a0633ed26b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:31.023278 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b93e08-0c1b-4b86-8765-a0633ed26b18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:31.025809 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b93e08-0c1b-4b86-8765-a0633ed26b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:31.025809 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b93e08-0c1b-4b86-8765-a0633ed26b18 admin admin] PATH_INFO: `/v3/projects/bbf4b01c3cdd4b8f98406b488682f4aa/users/a2246d3375eb4581b0e9c0d2204d725d/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:31.030342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b93e08-0c1b-4b86-8765-a0633ed26b18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbf4b01c3cdd4b8f98406b488682f4aa, user_id=a2246d3375eb4581b0e9c0d2204d725d, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:31.038520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b93e08-0c1b-4b86-8765-a0633ed26b18 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:31.081922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1017/2012] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:30 2022] PUT /identity/v3/projects/bbf4b01c3cdd4b8f98406b488682f4aa/users/a2246d3375eb4581b0e9c0d2204d725d/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:31.089871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83661ac6-c56f-4c2f-82ed-186abc7a5b7d None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:31.090547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83661ac6-c56f-4c2f-82ed-186abc7a5b7d None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:31.090933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83661ac6-c56f-4c2f-82ed-186abc7a5b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:31.159499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-83661ac6-c56f-4c2f-82ed-186abc7a5b7d None None] MFA Rules not processed for user `a2246d3375eb4581b0e9c0d2204d725d`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:31.162969 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-83661ac6-c56f-4c2f-82ed-186abc7a5b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:31.199539 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:31.230767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b93e08-0c1b-4b86-8765-a0633ed26b18 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:31.242080 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-33b93e08-0c1b-4b86-8765-a0633ed26b18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:31.288772 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:31.383287 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 996/2013] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:31 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:31.399414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83661ac6-c56f-4c2f-82ed-186abc7a5b7d None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:31.402647 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-83661ac6-c56f-4c2f-82ed-186abc7a5b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:31.435519 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b93e08-0c1b-4b86-8765-a0633ed26b18 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:31.449671 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3093f-91f1-415b-9b75-10ff97de991b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:31.450697 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3093f-91f1-415b-9b75-10ff97de991b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:31.450697 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3093f-91f1-415b-9b75-10ff97de991b admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:31.451017 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3093f-91f1-415b-9b75-10ff97de991b admin admin] PATH_INFO: `/v3/projects/900adb03cf9941c1be25fba21adcff7b` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:31.452758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3093f-91f1-415b-9b75-10ff97de991b admin admin] RBAC: Authorizing `identity:delete_project(project_id=900adb03cf9941c1be25fba21adcff7b)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:31.454890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3093f-91f1-415b-9b75-10ff97de991b admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:31.527061 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83661ac6-c56f-4c2f-82ed-186abc7a5b7d None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:31.547773 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac464b4-b952-46a6-a2bc-c63658c552d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:31.553494 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac464b4-b952-46a6-a2bc-c63658c552d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:31.553829 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac464b4-b952-46a6-a2bc-c63658c552d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:31.554102 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac464b4-b952-46a6-a2bc-c63658c552d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:31.578677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-3ff3093f-91f1-415b-9b75-10ff97de991b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '900adb03cf9941c1be25fba21adcff7b', 'request_id': 'req-3ff3093f-91f1-415b-9b75-10ff97de991b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:31.592702 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac464b4-b952-46a6-a2bc-c63658c552d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:31.595731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac464b4-b952-46a6-a2bc-c63658c552d8 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:31.611887 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-3ff3093f-91f1-415b-9b75-10ff97de991b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '900adb03cf9941c1be25fba21adcff7b', 'request_id': 'req-3ff3093f-91f1-415b-9b75-10ff97de991b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:31.647701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-3ff3093f-91f1-415b-9b75-10ff97de991b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '900adb03cf9941c1be25fba21adcff7b', 'request_id': 'req-3ff3093f-91f1-415b-9b75-10ff97de991b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:31.650634 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 997/2014] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:31 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:31.694197 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-3ff3093f-91f1-415b-9b75-10ff97de991b admin admin] The token cache is being invalidate because project 900adb03cf9941c1be25fba21adcff7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:31.697247 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-3ff3093f-91f1-415b-9b75-10ff97de991b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:31.704432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-3ff3093f-91f1-415b-9b75-10ff97de991b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:31.708233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1018/2015] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:14:31 2022] DELETE /identity/v3/projects/900adb03cf9941c1be25fba21adcff7b => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:31.720856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f76e38-8091-4bae-83bf-46306c042643 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:31.722163 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f76e38-8091-4bae-83bf-46306c042643 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:31.722718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f76e38-8091-4bae-83bf-46306c042643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:31.797463 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-f2f76e38-8091-4bae-83bf-46306c042643 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:31.800438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-f2f76e38-8091-4bae-83bf-46306c042643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:32.034552 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 998/2016] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:32.072277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e602e49-06a9-4112-bc4d-6577a73a0ec0 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:32.079065 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e602e49-06a9-4112-bc4d-6577a73a0ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:32.079600 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e602e49-06a9-4112-bc4d-6577a73a0ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:32.143844 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-0e602e49-06a9-4112-bc4d-6577a73a0ec0 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:32.158314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0e602e49-06a9-4112-bc4d-6577a73a0ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:32.187998 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1019/2017] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:32.196969 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f76e38-8091-4bae-83bf-46306c042643 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:32.206054 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e602e49-06a9-4112-bc4d-6577a73a0ec0 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:32.210779 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0e602e49-06a9-4112-bc4d-6577a73a0ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:32.219779 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f76e38-8091-4bae-83bf-46306c042643 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:32.242764 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7890e-ab8e-4ca4-a34a-e31a1caaa331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:32.248698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7890e-ab8e-4ca4-a34a-e31a1caaa331 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:32.249133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7890e-ab8e-4ca4-a34a-e31a1caaa331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:32.249857 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7890e-ab8e-4ca4-a34a-e31a1caaa331 admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:32.253218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7890e-ab8e-4ca4-a34a-e31a1caaa331 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:32.253218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7890e-ab8e-4ca4-a34a-e31a1caaa331 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:32.268733 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7890e-ab8e-4ca4-a34a-e31a1caaa331 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:32.303810 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 999/2018] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:32.311535 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7890e-ab8e-4ca4-a34a-e31a1caaa331 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:32.337848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7890e-ab8e-4ca4-a34a-e31a1caaa331 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:32.357851 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa421414-fbd9-4dd6-a9dd-e47379a0d099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:32.359213 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa421414-fbd9-4dd6-a9dd-e47379a0d099 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:32.359733 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa421414-fbd9-4dd6-a9dd-e47379a0d099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:32.360216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa421414-fbd9-4dd6-a9dd-e47379a0d099 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:32.361532 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa421414-fbd9-4dd6-a9dd-e47379a0d099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:32.365347 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa421414-fbd9-4dd6-a9dd-e47379a0d099 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:32.386669 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e602e49-06a9-4112-bc4d-6577a73a0ec0 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:32.392037 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cf470-8bb1-431b-9cac-894e0869f3e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:32.392907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cf470-8bb1-431b-9cac-894e0869f3e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:32.393469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cf470-8bb1-431b-9cac-894e0869f3e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:32.394528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cf470-8bb1-431b-9cac-894e0869f3e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:32.397309 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-be5cf470-8bb1-431b-9cac-894e0869f3e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:32.497386 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1000/2019] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:32 2022] POST /identity/v3/projects => generated 377 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:32.499371 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa421414-fbd9-4dd6-a9dd-e47379a0d099 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:32.527214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:32.534903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5cf470-8bb1-431b-9cac-894e0869f3e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:32.537822 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5cf470-8bb1-431b-9cac-894e0869f3e9 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:32.540705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:32.557034 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1020/2020] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:32 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:32.734318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa421414-fbd9-4dd6-a9dd-e47379a0d099 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:32.765637 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d492f4f-da32-4c3a-b18b-6cec9dd9296a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:32.766544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d492f4f-da32-4c3a-b18b-6cec9dd9296a admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:32.767294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d492f4f-da32-4c3a-b18b-6cec9dd9296a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:32.767750 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d492f4f-da32-4c3a-b18b-6cec9dd9296a admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:32.768750 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d492f4f-da32-4c3a-b18b-6cec9dd9296a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:32.772031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d492f4f-da32-4c3a-b18b-6cec9dd9296a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:32.838740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1001/2021] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:32 2022] POST /identity/v3/users => generated 289 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:32.845854 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cf470-8bb1-431b-9cac-894e0869f3e9 service cinder] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:32.873310 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cf470-8bb1-431b-9cac-894e0869f3e9 service cinder] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:32.892947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9b7cf-e627-4e33-a51a-4203fb90223a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:32.894070 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9b7cf-e627-4e33-a51a-4203fb90223a admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:32.894526 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9b7cf-e627-4e33-a51a-4203fb90223a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:32.895076 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9b7cf-e627-4e33-a51a-4203fb90223a admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:32.897671 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9b7cf-e627-4e33-a51a-4203fb90223a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:32.901451 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9b7cf-e627-4e33-a51a-4203fb90223a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:32.940718 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1021/2022] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:32.942020 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d492f4f-da32-4c3a-b18b-6cec9dd9296a admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:32.993785 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d492f4f-da32-4c3a-b18b-6cec9dd9296a admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:33.035385 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affc38c-7e86-43bb-b389-861c1cf1ff86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:33.038759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affc38c-7e86-43bb-b389-861c1cf1ff86 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:33.043143 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affc38c-7e86-43bb-b389-861c1cf1ff86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:33.044034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affc38c-7e86-43bb-b389-861c1cf1ff86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:33.045935 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affc38c-7e86-43bb-b389-861c1cf1ff86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:33.050187 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affc38c-7e86-43bb-b389-861c1cf1ff86 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:33.088859 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9b7cf-e627-4e33-a51a-4203fb90223a admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:33.092642 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1002/2023] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:33.135132 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9b7cf-e627-4e33-a51a-4203fb90223a admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:33.161438 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011aad08-ed98-4784-a09b-0cebf8d7a115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:33.163347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011aad08-ed98-4784-a09b-0cebf8d7a115 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:33.166987 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011aad08-ed98-4784-a09b-0cebf8d7a115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:33.166987 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011aad08-ed98-4784-a09b-0cebf8d7a115 admin admin] PATH_INFO: `/v3/projects/5056a3ecae834c8fbb3baea7c1e67c14/users/14360e05265c4ec580123dc9946ba6a8/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:33.265434 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011aad08-ed98-4784-a09b-0cebf8d7a115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5056a3ecae834c8fbb3baea7c1e67c14, user_id=14360e05265c4ec580123dc9946ba6a8, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:33.270483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011aad08-ed98-4784-a09b-0cebf8d7a115 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:33.376632 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1022/2024] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:33 2022] PUT /identity/v3/projects/5056a3ecae834c8fbb3baea7c1e67c14/users/14360e05265c4ec580123dc9946ba6a8/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:33.393456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8ca1d-357e-4176-afb2-c1617901f926 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:33.393924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8ca1d-357e-4176-afb2-c1617901f926 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:33.394973 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8ca1d-357e-4176-afb2-c1617901f926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:33.513849 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-b1c8ca1d-357e-4176-afb2-c1617901f926 None None] MFA Rules not processed for user `14360e05265c4ec580123dc9946ba6a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:33.523718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b1c8ca1d-357e-4176-afb2-c1617901f926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:33.803309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1003/2025] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:33.811383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011aad08-ed98-4784-a09b-0cebf8d7a115 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:34.001286 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011aad08-ed98-4784-a09b-0cebf8d7a115 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:34.011006 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735aeaa-59d8-4542-9b1f-cddd84d42942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:34.012334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735aeaa-59d8-4542-9b1f-cddd84d42942 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:34.013066 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735aeaa-59d8-4542-9b1f-cddd84d42942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:34.013779 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735aeaa-59d8-4542-9b1f-cddd84d42942 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:34.015610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735aeaa-59d8-4542-9b1f-cddd84d42942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:34.020114 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735aeaa-59d8-4542-9b1f-cddd84d42942 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:34.128770 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1023/2026] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:33 2022] POST /identity/v3/projects => generated 375 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:34.131897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8ca1d-357e-4176-afb2-c1617901f926 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:34.166016 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8ca1d-357e-4176-afb2-c1617901f926 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:34.369667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3ef29-7276-497e-a0b0-d755bf101aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:34.372033 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3ef29-7276-497e-a0b0-d755bf101aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:34.373087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3ef29-7276-497e-a0b0-d755bf101aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:34.374092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3ef29-7276-497e-a0b0-d755bf101aa2 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:34.376384 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d3ef29-7276-497e-a0b0-d755bf101aa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:34.381092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d3ef29-7276-497e-a0b0-d755bf101aa2 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:34.484121 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735aeaa-59d8-4542-9b1f-cddd84d42942 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:34.524879 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1004/2027] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:34 2022] POST /identity/v3/users => generated 288 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:34.543028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735aeaa-59d8-4542-9b1f-cddd84d42942 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:34.556182 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007ab31-8954-467b-a5cd-ef7055e908a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:34.558917 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007ab31-8954-467b-a5cd-ef7055e908a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:34.559858 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007ab31-8954-467b-a5cd-ef7055e908a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:34.562258 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007ab31-8954-467b-a5cd-ef7055e908a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:34.564119 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007ab31-8954-467b-a5cd-ef7055e908a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:34.571758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007ab31-8954-467b-a5cd-ef7055e908a6 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:34.592992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1024/2028] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:34.607117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3ef29-7276-497e-a0b0-d755bf101aa2 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:34.638170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3ef29-7276-497e-a0b0-d755bf101aa2 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:34.656882 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78536ed4-9395-4d97-bf34-b8b971b63517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:34.657853 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78536ed4-9395-4d97-bf34-b8b971b63517 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:34.658252 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78536ed4-9395-4d97-bf34-b8b971b63517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:34.658651 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78536ed4-9395-4d97-bf34-b8b971b63517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:34.659662 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78536ed4-9395-4d97-bf34-b8b971b63517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:34.661681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78536ed4-9395-4d97-bf34-b8b971b63517 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:34.684313 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:34.694937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1005/2029] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:14:34.697528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007ab31-8954-467b-a5cd-ef7055e908a6 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:34.735515 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007ab31-8954-467b-a5cd-ef7055e908a6 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:34.763731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d702e-f90c-4f23-922b-9886034eb56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:34.764794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d702e-f90c-4f23-922b-9886034eb56c admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:34.765377 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d702e-f90c-4f23-922b-9886034eb56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:34.765956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d702e-f90c-4f23-922b-9886034eb56c admin admin] PATH_INFO: `/v3/projects/ad39e84f611a4f13bb145c11663e9f76/users/e181d37ce3df4097abd7b5d59a9651f7/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:34.840696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d702e-f90c-4f23-922b-9886034eb56c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad39e84f611a4f13bb145c11663e9f76, user_id=e181d37ce3df4097abd7b5d59a9651f7, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:34.847924 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d702e-f90c-4f23-922b-9886034eb56c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:34.870492 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de318d-f404-401f-b5a2-4862aa9e5e08 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:34.870492 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de318d-f404-401f-b5a2-4862aa9e5e08 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:34.870785 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de318d-f404-401f-b5a2-4862aa9e5e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:34.881777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1025/2030] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:34 2022] PUT /identity/v3/projects/ad39e84f611a4f13bb145c11663e9f76/users/e181d37ce3df4097abd7b5d59a9651f7/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:14:34.911067 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb3bbd-3adc-40a2-83cd-48e525eb90bc None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:34.912992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb3bbd-3adc-40a2-83cd-48e525eb90bc None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:34.914589 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb3bbd-3adc-40a2-83cd-48e525eb90bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:34.968724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-80de318d-f404-401f-b5a2-4862aa9e5e08 None None] MFA Rules not processed for user `9945e4ffbc2548d5962d581aa905bb5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:34.970954 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-80de318d-f404-401f-b5a2-4862aa9e5e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:35.033619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-d1bb3bbd-3adc-40a2-83cd-48e525eb90bc None None] MFA Rules not processed for user `e181d37ce3df4097abd7b5d59a9651f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:35.035623 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-d1bb3bbd-3adc-40a2-83cd-48e525eb90bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:35.117553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1006/2031] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:35.119656 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de318d-f404-401f-b5a2-4862aa9e5e08 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:35.121777 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-80de318d-f404-401f-b5a2-4862aa9e5e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:35.194001 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:35.208243 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1026/2032] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:35.217622 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb3bbd-3adc-40a2-83cd-48e525eb90bc None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:35.301002 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de318d-f404-401f-b5a2-4862aa9e5e08 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:35.310728 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e925f9d-4259-4afa-8d91-bcfc7093bf34 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4461a389da5848f5afd9854758707829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:35.314286 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e925f9d-4259-4afa-8d91-bcfc7093bf34 service glance] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:35.315069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e925f9d-4259-4afa-8d91-bcfc7093bf34 service glance] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:35.315734 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e925f9d-4259-4afa-8d91-bcfc7093bf34 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:35.320876 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-3e925f9d-4259-4afa-8d91-bcfc7093bf34 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:35.403184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb3bbd-3adc-40a2-83cd-48e525eb90bc None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:35.413095 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e6b67-9709-48bb-90a9-0d2a7ed3bf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:35.414945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e6b67-9709-48bb-90a9-0d2a7ed3bf7a admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:35.415518 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e6b67-9709-48bb-90a9-0d2a7ed3bf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:35.416301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e6b67-9709-48bb-90a9-0d2a7ed3bf7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:35.417470 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e6b67-9709-48bb-90a9-0d2a7ed3bf7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:35.421418 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e6b67-9709-48bb-90a9-0d2a7ed3bf7a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:35.462213 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e925f9d-4259-4afa-8d91-bcfc7093bf34 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:35.465006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e925f9d-4259-4afa-8d91-bcfc7093bf34 service glance] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:35.479857 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1007/2033] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:35.497117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1027/2034] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:35 2022] POST /identity/v3/projects => generated 375 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:35.499491 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e925f9d-4259-4afa-8d91-bcfc7093bf34 service glance] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:35.523529 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e925f9d-4259-4afa-8d91-bcfc7093bf34 service glance] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:35.653374 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:35.708815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecd86a-ca1a-4d8c-94cc-24e014ffe2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:35.710049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecd86a-ca1a-4d8c-94cc-24e014ffe2be admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:35.710549 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecd86a-ca1a-4d8c-94cc-24e014ffe2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:35.711735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecd86a-ca1a-4d8c-94cc-24e014ffe2be admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:35.713075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecd86a-ca1a-4d8c-94cc-24e014ffe2be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:35.717070 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecd86a-ca1a-4d8c-94cc-24e014ffe2be admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:35.758794 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1008/2035] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:35 2022] POST /identity/v3/users => generated 294 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:14:35.765608 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e6b67-9709-48bb-90a9-0d2a7ed3bf7a admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:35.781529 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c03ed-afb8-4cfb-bcba-b3bc69725aba None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:35.782278 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c03ed-afb8-4cfb-bcba-b3bc69725aba None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:35.782931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c03ed-afb8-4cfb-bcba-b3bc69725aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:35.787879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e6b67-9709-48bb-90a9-0d2a7ed3bf7a admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:35.794719 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620396aa-863a-4884-9239-1458645f0dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:35.795576 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620396aa-863a-4884-9239-1458645f0dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:35.795951 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620396aa-863a-4884-9239-1458645f0dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:35.796330 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620396aa-863a-4884-9239-1458645f0dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:35.797243 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620396aa-863a-4884-9239-1458645f0dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:35.799166 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620396aa-863a-4884-9239-1458645f0dd5 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:35.818134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1028/2036] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:35.825812 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620396aa-863a-4884-9239-1458645f0dd5 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:35.834725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-306c03ed-afb8-4cfb-bcba-b3bc69725aba None None] MFA Rules not processed for user `4370f8f474c2446e989d0d8c5b75e87a`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:35.837326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-306c03ed-afb8-4cfb-bcba-b3bc69725aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:35.844324 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620396aa-863a-4884-9239-1458645f0dd5 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:35.850891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5ed6b-877f-42e7-ac1f-7bb32ab7cc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:35.852697 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5ed6b-877f-42e7-ac1f-7bb32ab7cc58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:35.853023 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5ed6b-877f-42e7-ac1f-7bb32ab7cc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:35.853365 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5ed6b-877f-42e7-ac1f-7bb32ab7cc58 admin admin] PATH_INFO: `/v3/projects/1567188877984156bb45e9f014ad2c24/users/c629935f44cf4d6fb51c57d118b61d14/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:35.883682 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5ed6b-877f-42e7-ac1f-7bb32ab7cc58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1567188877984156bb45e9f014ad2c24, user_id=c629935f44cf4d6fb51c57d118b61d14, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:35.891741 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5ed6b-877f-42e7-ac1f-7bb32ab7cc58 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:35.915724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1029/2037] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:35 2022] PUT /identity/v3/projects/1567188877984156bb45e9f014ad2c24/users/c629935f44cf4d6fb51c57d118b61d14/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:35.920655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5ed6b-877f-42e7-ac1f-7bb32ab7cc58 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:35.985530 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1009/2038] 10.222.0.93 () {58 vars in 1009 bytes} [Mon Aug 1 15:14:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:36.069227 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c03ed-afb8-4cfb-bcba-b3bc69725aba None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:36.071446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-306c03ed-afb8-4cfb-bcba-b3bc69725aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:36.072624 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5ed6b-877f-42e7-ac1f-7bb32ab7cc58 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:36.085419 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ca106-2170-44a0-a088-1be8359e8761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:36.087227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ca106-2170-44a0-a088-1be8359e8761 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:36.087900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ca106-2170-44a0-a088-1be8359e8761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:36.088514 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ca106-2170-44a0-a088-1be8359e8761 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:36.090046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ca106-2170-44a0-a088-1be8359e8761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:36.093518 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ca106-2170-44a0-a088-1be8359e8761 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:36.121442 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1030/2039] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:36.124907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ca106-2170-44a0-a088-1be8359e8761 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:36.148898 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ca106-2170-44a0-a088-1be8359e8761 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:36.158567 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4770d-f3b4-4a34-be5f-a67dc92aad56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:36.159757 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4770d-f3b4-4a34-be5f-a67dc92aad56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:36.160106 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4770d-f3b4-4a34-be5f-a67dc92aad56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:36.160618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4770d-f3b4-4a34-be5f-a67dc92aad56 admin admin] PATH_INFO: `/v3/projects/1567188877984156bb45e9f014ad2c24/users/c629935f44cf4d6fb51c57d118b61d14/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:36.163694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4770d-f3b4-4a34-be5f-a67dc92aad56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1567188877984156bb45e9f014ad2c24, user_id=c629935f44cf4d6fb51c57d118b61d14, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:36.168842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4770d-f3b4-4a34-be5f-a67dc92aad56 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:36.188311 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1031/2040] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:36 2022] PUT /identity/v3/projects/1567188877984156bb45e9f014ad2c24/users/c629935f44cf4d6fb51c57d118b61d14/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:14:36.206242 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c03ed-afb8-4cfb-bcba-b3bc69725aba None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:36.208072 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39cda5-477e-46f4-8d53-f2b8fcf046d5 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:36.208883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39cda5-477e-46f4-8d53-f2b8fcf046d5 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:36.209349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39cda5-477e-46f4-8d53-f2b8fcf046d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:36.267688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-fa39cda5-477e-46f4-8d53-f2b8fcf046d5 None None] MFA Rules not processed for user `c629935f44cf4d6fb51c57d118b61d14`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:36.270346 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-fa39cda5-477e-46f4-8d53-f2b8fcf046d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:36.330874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:36.383404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ef202-85a8-4b8f-a632-71068f88b057 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4df37b02152c48509243b73ebbf49d8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:36.387082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ef202-85a8-4b8f-a632-71068f88b057 service swift] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:36.387082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ef202-85a8-4b8f-a632-71068f88b057 service swift] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:36.387082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ef202-85a8-4b8f-a632-71068f88b057 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:36.450501 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1032/2041] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:36.478134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:36.480684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39cda5-477e-46f4-8d53-f2b8fcf046d5 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:36.486050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-fa39cda5-477e-46f4-8d53-f2b8fcf046d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:36.651006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ef202-85a8-4b8f-a632-71068f88b057 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:36.654231 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ef202-85a8-4b8f-a632-71068f88b057 service swift] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:36.683763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39cda5-477e-46f4-8d53-f2b8fcf046d5 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:36.696356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2713a-4660-47a3-a9c5-3741c9ffe580 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:36.698046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2713a-4660-47a3-a9c5-3741c9ffe580 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:36.699344 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2713a-4660-47a3-a9c5-3741c9ffe580 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:36.702696 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1010/2042] 10.222.0.93 () {60 vars in 1384 bytes} [Mon Aug 1 15:14:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 15:14:36.714171 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2713a-4660-47a3-a9c5-3741c9ffe580 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:36.905301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d2713a-4660-47a3-a9c5-3741c9ffe580 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:36.908341 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d2713a-4660-47a3-a9c5-3741c9ffe580 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:36.937472 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1033/2043] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:41.366037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ef202-85a8-4b8f-a632-71068f88b057 service swift] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:41.378161 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ef202-85a8-4b8f-a632-71068f88b057 service swift] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:41.397816 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4f313-e66f-47d9-8fa7-3c9845884648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:41.398819 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4f313-e66f-47d9-8fa7-3c9845884648 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:41.399269 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4f313-e66f-47d9-8fa7-3c9845884648 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:41.399709 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4f313-e66f-47d9-8fa7-3c9845884648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:41.565006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4f313-e66f-47d9-8fa7-3c9845884648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:41.566629 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4f313-e66f-47d9-8fa7-3c9845884648 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:41.628429 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1011/2044] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:42.390783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2713a-4660-47a3-a9c5-3741c9ffe580 service cinder] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:42.392672 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a4d2713a-4660-47a3-a9c5-3741c9ffe580 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:42.684919 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2713a-4660-47a3-a9c5-3741c9ffe580 service cinder] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:42.693803 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43731b-084f-4208-a221-d243be593bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:42.695381 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43731b-084f-4208-a221-d243be593bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:42.696133 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43731b-084f-4208-a221-d243be593bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:42.697490 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43731b-084f-4208-a221-d243be593bc2 admin admin] PATH_INFO: `/v3/users/18a22909a8a14106bbc8e5e09fab7bd5` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:42.700289 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43731b-084f-4208-a221-d243be593bc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18a22909a8a14106bbc8e5e09fab7bd5)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:42.706015 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43731b-084f-4208-a221-d243be593bc2 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:42.740430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:42.911343 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4b43731b-084f-4208-a221-d243be593bc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18a22909a8a14106bbc8e5e09fab7bd5'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:42.971694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4b43731b-084f-4208-a221-d243be593bc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18a22909a8a14106bbc8e5e09fab7bd5'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:42.996176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4b43731b-084f-4208-a221-d243be593bc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18a22909a8a14106bbc8e5e09fab7bd5'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:42.998302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4b43731b-084f-4208-a221-d243be593bc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18a22909a8a14106bbc8e5e09fab7bd5'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:43.000366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4b43731b-084f-4208-a221-d243be593bc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18a22909a8a14106bbc8e5e09fab7bd5'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:43.059783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1034/2045] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:14:42 2022] DELETE /identity/v3/users/18a22909a8a14106bbc8e5e09fab7bd5 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:14:43.066467 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676be7b4-b1da-451f-b420-4a14aec19863 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:43.068973 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676be7b4-b1da-451f-b420-4a14aec19863 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:43.070016 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676be7b4-b1da-451f-b420-4a14aec19863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:43.173143 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-676be7b4-b1da-451f-b420-4a14aec19863 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:43.177115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-676be7b4-b1da-451f-b420-4a14aec19863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:43.384390 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1012/2046] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:43.410579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43731b-084f-4208-a221-d243be593bc2 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:43.414253 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4b43731b-084f-4208-a221-d243be593bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:43.593146 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676be7b4-b1da-451f-b420-4a14aec19863 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:43.594930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-676be7b4-b1da-451f-b420-4a14aec19863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:43.600083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43731b-084f-4208-a221-d243be593bc2 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:43.607429 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9312df-bb52-4d99-af3e-4073f9a6fa33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:43.609279 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9312df-bb52-4d99-af3e-4073f9a6fa33 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:43.610000 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9312df-bb52-4d99-af3e-4073f9a6fa33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:43.610613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9312df-bb52-4d99-af3e-4073f9a6fa33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:43.629401 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9312df-bb52-4d99-af3e-4073f9a6fa33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:43.633059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9312df-bb52-4d99-af3e-4073f9a6fa33 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:43.664808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1035/2047] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:43.697752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676be7b4-b1da-451f-b420-4a14aec19863 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:43.714013 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f85ac9-51fd-48fc-8e6f-fba14921bfaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:43.715040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f85ac9-51fd-48fc-8e6f-fba14921bfaf service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:43.715397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f85ac9-51fd-48fc-8e6f-fba14921bfaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:43.715763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f85ac9-51fd-48fc-8e6f-fba14921bfaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:43.722298 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-15f85ac9-51fd-48fc-8e6f-fba14921bfaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:43.749557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:43.905809 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f85ac9-51fd-48fc-8e6f-fba14921bfaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:43.907475 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f85ac9-51fd-48fc-8e6f-fba14921bfaf service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:43.938264 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1013/2048] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:43 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:14:44.529126 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9312df-bb52-4d99-af3e-4073f9a6fa33 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:44.531805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5f9312df-bb52-4d99-af3e-4073f9a6fa33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:44.569183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f85ac9-51fd-48fc-8e6f-fba14921bfaf service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:44.575794 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-15f85ac9-51fd-48fc-8e6f-fba14921bfaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:44.600644 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9312df-bb52-4d99-af3e-4073f9a6fa33 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:44.615087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e9727-027c-4751-90af-ad99c5a52260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:44.616213 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e9727-027c-4751-90af-ad99c5a52260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:44.617290 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e9727-027c-4751-90af-ad99c5a52260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:44.617775 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e9727-027c-4751-90af-ad99c5a52260 admin admin] PATH_INFO: `/v3/users/2d67bc3050ce4fc2bfab6d2f8e578604` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:44.620214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e9727-027c-4751-90af-ad99c5a52260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d67bc3050ce4fc2bfab6d2f8e578604)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:44.623883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e9727-027c-4751-90af-ad99c5a52260 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:44.631046 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f85ac9-51fd-48fc-8e6f-fba14921bfaf service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:44.640619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a154f7-2138-4080-9c39-7d11f316022d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:44.642577 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a154f7-2138-4080-9c39-7d11f316022d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:44.642891 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a154f7-2138-4080-9c39-7d11f316022d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:44.643146 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a154f7-2138-4080-9c39-7d11f316022d admin admin] PATH_INFO: `/v3/projects/b6a675927b28435295bdf40424a01ec5` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:44.645506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a154f7-2138-4080-9c39-7d11f316022d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6a675927b28435295bdf40424a01ec5)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:44.648316 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a154f7-2138-4080-9c39-7d11f316022d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:44.732437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-532e9727-027c-4751-90af-ad99c5a52260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d67bc3050ce4fc2bfab6d2f8e578604'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:44.747732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-532e9727-027c-4751-90af-ad99c5a52260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d67bc3050ce4fc2bfab6d2f8e578604'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:44.760696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-532e9727-027c-4751-90af-ad99c5a52260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d67bc3050ce4fc2bfab6d2f8e578604'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:44.762153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-532e9727-027c-4751-90af-ad99c5a52260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d67bc3050ce4fc2bfab6d2f8e578604'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:44.763206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-532e9727-027c-4751-90af-ad99c5a52260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d67bc3050ce4fc2bfab6d2f8e578604'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:44.771954 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-56a154f7-2138-4080-9c39-7d11f316022d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6a675927b28435295bdf40424a01ec5', 'request_id': 'req-56a154f7-2138-4080-9c39-7d11f316022d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:44.781457 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-56a154f7-2138-4080-9c39-7d11f316022d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6a675927b28435295bdf40424a01ec5', 'request_id': 'req-56a154f7-2138-4080-9c39-7d11f316022d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:44.791630 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1036/2049] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:14:44 2022] DELETE /identity/v3/users/2d67bc3050ce4fc2bfab6d2f8e578604 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:44.797481 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79dfab-c0fa-4dc2-b85e-bbbe35a7d082 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:44.797827 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79dfab-c0fa-4dc2-b85e-bbbe35a7d082 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:44.798222 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79dfab-c0fa-4dc2-b85e-bbbe35a7d082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:44.809964 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:44.816745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-56a154f7-2138-4080-9c39-7d11f316022d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6a675927b28435295bdf40424a01ec5', 'request_id': 'req-56a154f7-2138-4080-9c39-7d11f316022d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:44.837247 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-4f79dfab-c0fa-4dc2-b85e-bbbe35a7d082 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:44.838956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4f79dfab-c0fa-4dc2-b85e-bbbe35a7d082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:44.860024 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-56a154f7-2138-4080-9c39-7d11f316022d admin admin] The token cache is being invalidate because project b6a675927b28435295bdf40424a01ec5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:44.860544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-56a154f7-2138-4080-9c39-7d11f316022d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:44.861837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-56a154f7-2138-4080-9c39-7d11f316022d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:44.865187 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1014/2050] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:14:44 2022] DELETE /identity/v3/projects/b6a675927b28435295bdf40424a01ec5 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:44.875371 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f615ed-505f-4ccc-8a44-7ff42dbf2dcc None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:44.881161 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f615ed-505f-4ccc-8a44-7ff42dbf2dcc None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:44.882646 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f615ed-505f-4ccc-8a44-7ff42dbf2dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:44.925742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-90f615ed-505f-4ccc-8a44-7ff42dbf2dcc None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:44.927236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-90f615ed-505f-4ccc-8a44-7ff42dbf2dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:44.950954 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1037/2051] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:44.990494 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79dfab-c0fa-4dc2-b85e-bbbe35a7d082 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:44.993507 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4f79dfab-c0fa-4dc2-b85e-bbbe35a7d082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:45.063964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1015/2052] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:45.065587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b69828-8f7b-48d4-9fdf-1715fbe2977e None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:45.066078 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b69828-8f7b-48d4-9fdf-1715fbe2977e None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:45.066408 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b69828-8f7b-48d4-9fdf-1715fbe2977e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:45.102995 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-98b69828-8f7b-48d4-9fdf-1715fbe2977e None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:45.105365 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-98b69828-8f7b-48d4-9fdf-1715fbe2977e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:45.119979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1016/2053] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:45.125884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b69828-8f7b-48d4-9fdf-1715fbe2977e None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:45.154753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b69828-8f7b-48d4-9fdf-1715fbe2977e None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:45.161383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79dfab-c0fa-4dc2-b85e-bbbe35a7d082 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:45.162243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97fab2-9fe8-432a-8841-c116640859db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:45.162896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97fab2-9fe8-432a-8841-c116640859db admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:45.163177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97fab2-9fe8-432a-8841-c116640859db admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:45.164504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97fab2-9fe8-432a-8841-c116640859db admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:45.165193 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97fab2-9fe8-432a-8841-c116640859db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:45.165590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97fab2-9fe8-432a-8841-c116640859db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:45.167179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97fab2-9fe8-432a-8841-c116640859db admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:45.185934 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c89b2b-be87-4ae4-9cb6-815868604e9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:45.190278 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1017/2054] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:45.191461 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97fab2-9fe8-432a-8841-c116640859db admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:45.192896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c89b2b-be87-4ae4-9cb6-815868604e9e service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:45.197148 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c89b2b-be87-4ae4-9cb6-815868604e9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:45.198934 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c89b2b-be87-4ae4-9cb6-815868604e9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:45.208498 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97fab2-9fe8-432a-8841-c116640859db admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:45.219955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b135d6-50ef-40c8-b0d8-6c76a452abd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:45.221023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b135d6-50ef-40c8-b0d8-6c76a452abd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:45.223473 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b135d6-50ef-40c8-b0d8-6c76a452abd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:45.224155 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b135d6-50ef-40c8-b0d8-6c76a452abd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:45.225389 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b135d6-50ef-40c8-b0d8-6c76a452abd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:45.230123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b135d6-50ef-40c8-b0d8-6c76a452abd9 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:45.235886 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c89b2b-be87-4ae4-9cb6-815868604e9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:45.237710 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c89b2b-be87-4ae4-9cb6-815868604e9e service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:45.255935 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:45.264516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1038/2055] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:14:45.330800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1018/2056] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:45 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:45.336642 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c89b2b-be87-4ae4-9cb6-815868604e9e service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:45.364515 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c89b2b-be87-4ae4-9cb6-815868604e9e service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:45.525758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe40086-5aa7-4547-b12e-269d302f8b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:45.526982 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe40086-5aa7-4547-b12e-269d302f8b53 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:45.526982 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe40086-5aa7-4547-b12e-269d302f8b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:45.527726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe40086-5aa7-4547-b12e-269d302f8b53 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:45.529723 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe40086-5aa7-4547-b12e-269d302f8b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:45.534119 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe40086-5aa7-4547-b12e-269d302f8b53 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:45.623083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1039/2057] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:45 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:45.623778 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b135d6-50ef-40c8-b0d8-6c76a452abd9 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:45.640024 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b135d6-50ef-40c8-b0d8-6c76a452abd9 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:45.656124 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d86bc-d021-4f2d-9925-be4e8c00c6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:45.656822 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d86bc-d021-4f2d-9925-be4e8c00c6f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:45.657483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d86bc-d021-4f2d-9925-be4e8c00c6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:45.657766 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d86bc-d021-4f2d-9925-be4e8c00c6f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:45.658493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d86bc-d021-4f2d-9925-be4e8c00c6f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:45.661393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d86bc-d021-4f2d-9925-be4e8c00c6f6 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:45.687634 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1019/2058] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:14:45.688931 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe40086-5aa7-4547-b12e-269d302f8b53 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:45.727985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe40086-5aa7-4547-b12e-269d302f8b53 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:45.790467 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b5a90-93aa-4765-86ad-2673622db6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:45.792688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b5a90-93aa-4765-86ad-2673622db6c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:45.800364 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b5a90-93aa-4765-86ad-2673622db6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:45.801277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b5a90-93aa-4765-86ad-2673622db6c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:45.803744 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b5a90-93aa-4765-86ad-2673622db6c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:45.810472 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b5a90-93aa-4765-86ad-2673622db6c0 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:45.852319 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1040/2059] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:45.857021 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d86bc-d021-4f2d-9925-be4e8c00c6f6 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:45.874859 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d86bc-d021-4f2d-9925-be4e8c00c6f6 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:45.894225 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225864ff-1b92-4977-b638-669d55e6c5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:45.896381 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225864ff-1b92-4977-b638-669d55e6c5b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:45.899373 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225864ff-1b92-4977-b638-669d55e6c5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:45.899973 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225864ff-1b92-4977-b638-669d55e6c5b0 admin admin] PATH_INFO: `/v3/projects/c09a0a659b0640c0884544172763e034/users/56a746907610432195d4bfdc6bc09937/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:45.952520 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225864ff-1b92-4977-b638-669d55e6c5b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c09a0a659b0640c0884544172763e034, user_id=56a746907610432195d4bfdc6bc09937, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:45.957286 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225864ff-1b92-4977-b638-669d55e6c5b0 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:45.998328 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1020/2060] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:45 2022] PUT /identity/v3/projects/c09a0a659b0640c0884544172763e034/users/56a746907610432195d4bfdc6bc09937/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:14:46.025631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7744ec-e5c3-4755-b9bf-752674232484 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:46.025921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7744ec-e5c3-4755-b9bf-752674232484 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:46.026207 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7744ec-e5c3-4755-b9bf-752674232484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:46.105726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-ef7744ec-e5c3-4755-b9bf-752674232484 None None] MFA Rules not processed for user `56a746907610432195d4bfdc6bc09937`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:46.107128 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ef7744ec-e5c3-4755-b9bf-752674232484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:46.127787 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:46.345028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225864ff-1b92-4977-b638-669d55e6c5b0 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:46.345474 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1041/2061] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:46.477153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225864ff-1b92-4977-b638-669d55e6c5b0 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:46.485436 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06fee3-2f8a-49e0-8a0b-1890bf8911d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:46.487578 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06fee3-2f8a-49e0-8a0b-1890bf8911d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:46.488242 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06fee3-2f8a-49e0-8a0b-1890bf8911d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:46.488930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06fee3-2f8a-49e0-8a0b-1890bf8911d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:46.493856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06fee3-2f8a-49e0-8a0b-1890bf8911d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:46.494900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06fee3-2f8a-49e0-8a0b-1890bf8911d3 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:46.526275 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7744ec-e5c3-4755-b9bf-752674232484 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:46.528893 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ef7744ec-e5c3-4755-b9bf-752674232484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:46.552790 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1021/2062] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:46 2022] POST /identity/v3/projects => generated 385 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:46.557635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06fee3-2f8a-49e0-8a0b-1890bf8911d3 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:46.616383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7744ec-e5c3-4755-b9bf-752674232484 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:46.713237 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06fee3-2f8a-49e0-8a0b-1890bf8911d3 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:46.732031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d372873-c843-4ac7-9934-b67fdc4ff19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:46.733386 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d372873-c843-4ac7-9934-b67fdc4ff19e admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:46.734045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d372873-c843-4ac7-9934-b67fdc4ff19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:46.734464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d372873-c843-4ac7-9934-b67fdc4ff19e admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:46.735430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d372873-c843-4ac7-9934-b67fdc4ff19e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:46.739567 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d372873-c843-4ac7-9934-b67fdc4ff19e admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:46.821149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1022/2063] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:46 2022] POST /identity/v3/users => generated 293 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:46.823098 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04bc67-f55f-4017-89ab-7dc47964fd37 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:46.823518 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04bc67-f55f-4017-89ab-7dc47964fd37 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:46.823913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04bc67-f55f-4017-89ab-7dc47964fd37 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:46.826169 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1023/2064] 10.222.0.93 () {54 vars in 921 bytes} [Mon Aug 1 15:14:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:14:46.827346 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04bc67-f55f-4017-89ab-7dc47964fd37 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:46.833030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-4d04bc67-f55f-4017-89ab-7dc47964fd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:46.861218 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20ffb2-e0b9-49c8-87ca-4005dd75b76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:46.861530 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04bc67-f55f-4017-89ab-7dc47964fd37 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:46.861862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20ffb2-e0b9-49c8-87ca-4005dd75b76c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:46.862115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20ffb2-e0b9-49c8-87ca-4005dd75b76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:46.862382 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20ffb2-e0b9-49c8-87ca-4005dd75b76c admin admin] PATH_INFO: `/v3/projects/49d083b0c8514ab2b458e0e8ad485bee` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:46.863701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20ffb2-e0b9-49c8-87ca-4005dd75b76c admin admin] RBAC: Authorizing `identity:delete_project(project_id=49d083b0c8514ab2b458e0e8ad485bee)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:46.865837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20ffb2-e0b9-49c8-87ca-4005dd75b76c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:46.871515 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa687c-e281-4096-a247-9a42d7238c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:46.872691 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa687c-e281-4096-a247-9a42d7238c5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:46.873137 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa687c-e281-4096-a247-9a42d7238c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:46.873615 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa687c-e281-4096-a247-9a42d7238c5a admin admin] PATH_INFO: `/v3/users/e708da2136a248c196ae4ecb531bb322` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:46.876320 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa687c-e281-4096-a247-9a42d7238c5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e708da2136a248c196ae4ecb531bb322)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:46.880826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa687c-e281-4096-a247-9a42d7238c5a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:46.935319 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=90,keystone:INSERT=6,keystone:DELETE=4 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:46.973408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5b20ffb2-e0b9-49c8-87ca-4005dd75b76c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49d083b0c8514ab2b458e0e8ad485bee', 'request_id': 'req-5b20ffb2-e0b9-49c8-87ca-4005dd75b76c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:46.990850 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5b20ffb2-e0b9-49c8-87ca-4005dd75b76c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49d083b0c8514ab2b458e0e8ad485bee', 'request_id': 'req-5b20ffb2-e0b9-49c8-87ca-4005dd75b76c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:47.002456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-f4fa687c-e281-4096-a247-9a42d7238c5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e708da2136a248c196ae4ecb531bb322'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:47.003444 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-f4fa687c-e281-4096-a247-9a42d7238c5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e708da2136a248c196ae4ecb531bb322'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:47.008209 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5b20ffb2-e0b9-49c8-87ca-4005dd75b76c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49d083b0c8514ab2b458e0e8ad485bee', 'request_id': 'req-5b20ffb2-e0b9-49c8-87ca-4005dd75b76c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:47.026599 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-f4fa687c-e281-4096-a247-9a42d7238c5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e708da2136a248c196ae4ecb531bb322'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:47.027480 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-f4fa687c-e281-4096-a247-9a42d7238c5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e708da2136a248c196ae4ecb531bb322'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:47.041159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5b20ffb2-e0b9-49c8-87ca-4005dd75b76c admin admin] The token cache is being invalidate because project 49d083b0c8514ab2b458e0e8ad485bee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:47.042131 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5b20ffb2-e0b9-49c8-87ca-4005dd75b76c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:47.044196 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5b20ffb2-e0b9-49c8-87ca-4005dd75b76c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:47.052246 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1042/2065] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:14:46 2022] DELETE /identity/v3/projects/49d083b0c8514ab2b458e0e8ad485bee => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:47.053898 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20ffb2-e0b9-49c8-87ca-4005dd75b76c admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:47.057261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5b20ffb2-e0b9-49c8-87ca-4005dd75b76c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:47.065154 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-f4fa687c-e281-4096-a247-9a42d7238c5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e708da2136a248c196ae4ecb531bb322'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:47.098933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1024/2066] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:14:46 2022] DELETE /identity/v3/users/e708da2136a248c196ae4ecb531bb322 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:47.098933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facd9d7-eb37-4cbf-879a-63df81e9b248 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:47.098933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facd9d7-eb37-4cbf-879a-63df81e9b248 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:47.098933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facd9d7-eb37-4cbf-879a-63df81e9b248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:47.131543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-3facd9d7-eb37-4cbf-879a-63df81e9b248 None None] MFA Rules not processed for user `a07e3e25e6a8466fb237dcb836aecda2`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:47.134093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-3facd9d7-eb37-4cbf-879a-63df81e9b248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:47.191448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:47.238137 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20ffb2-e0b9-49c8-87ca-4005dd75b76c admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:47.301193 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1025/2067] 10.222.0.93 () {58 vars in 1018 bytes} [Mon Aug 1 15:14:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:47.303316 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facd9d7-eb37-4cbf-879a-63df81e9b248 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:47.318193 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-3facd9d7-eb37-4cbf-879a-63df81e9b248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:47.335987 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81bdc9f-f20c-43f0-93d0-758095a860e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:47.337637 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bdc9f-f20c-43f0-93d0-758095a860e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:47.338059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bdc9f-f20c-43f0-93d0-758095a860e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:47.338488 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bdc9f-f20c-43f0-93d0-758095a860e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:47.339568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81bdc9f-f20c-43f0-93d0-758095a860e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:47.341776 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81bdc9f-f20c-43f0-93d0-758095a860e8 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:47.362594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1043/2068] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:47.362594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81bdc9f-f20c-43f0-93d0-758095a860e8 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:47.362594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b81bdc9f-f20c-43f0-93d0-758095a860e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:47.396384 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81bdc9f-f20c-43f0-93d0-758095a860e8 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:47.431755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58671ecc-4600-4ea3-8121-6a81212ed594 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07e3e25e6a8466fb237dcb836aecda2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:47.432409 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58671ecc-4600-4ea3-8121-6a81212ed594 service placement] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:47.432834 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58671ecc-4600-4ea3-8121-6a81212ed594 service placement] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:47.433116 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58671ecc-4600-4ea3-8121-6a81212ed594 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:47.462469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58671ecc-4600-4ea3-8121-6a81212ed594 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:47.466693 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58671ecc-4600-4ea3-8121-6a81212ed594 service placement] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:47.513280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1044/2069] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:47.514224 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58671ecc-4600-4ea3-8121-6a81212ed594 service placement] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:47.561659 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facd9d7-eb37-4cbf-879a-63df81e9b248 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:47.576783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c7895-8b35-446e-90b8-51f3a59c6719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:47.576783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c7895-8b35-446e-90b8-51f3a59c6719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:47.576783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c7895-8b35-446e-90b8-51f3a59c6719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:47.577325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58671ecc-4600-4ea3-8121-6a81212ed594 service placement] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:47.577466 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c7895-8b35-446e-90b8-51f3a59c6719 admin admin] PATH_INFO: `/v3/users/a2246d3375eb4581b0e9c0d2204d725d` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:47.577466 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c7895-8b35-446e-90b8-51f3a59c6719 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2246d3375eb4581b0e9c0d2204d725d)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:47.578095 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c7895-8b35-446e-90b8-51f3a59c6719 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:47.611546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35c52f-d37b-48c3-ad7b-228474d48a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:47.613023 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35c52f-d37b-48c3-ad7b-228474d48a3d admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:47.613535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35c52f-d37b-48c3-ad7b-228474d48a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:47.614018 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35c52f-d37b-48c3-ad7b-228474d48a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:47.615313 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c35c52f-d37b-48c3-ad7b-228474d48a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:47.633254 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c35c52f-d37b-48c3-ad7b-228474d48a3d admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:47.656907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:47.670189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-7a7c7895-8b35-446e-90b8-51f3a59c6719 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2246d3375eb4581b0e9c0d2204d725d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:47.671023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-7a7c7895-8b35-446e-90b8-51f3a59c6719 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2246d3375eb4581b0e9c0d2204d725d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:47.682251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-7a7c7895-8b35-446e-90b8-51f3a59c6719 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2246d3375eb4581b0e9c0d2204d725d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:47.683015 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1045/2070] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:47.683227 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-7a7c7895-8b35-446e-90b8-51f3a59c6719 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2246d3375eb4581b0e9c0d2204d725d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:47.685027 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35c52f-d37b-48c3-ad7b-228474d48a3d admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:47.693996 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5c35c52f-d37b-48c3-ad7b-228474d48a3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:47.707558 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-7a7c7895-8b35-446e-90b8-51f3a59c6719 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2246d3375eb4581b0e9c0d2204d725d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:47.740753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35c52f-d37b-48c3-ad7b-228474d48a3d admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:47.746124 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1026/2071] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:14:47 2022] DELETE /identity/v3/users/a2246d3375eb4581b0e9c0d2204d725d => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:47.747435 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c7895-8b35-446e-90b8-51f3a59c6719 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:47.751835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-7a7c7895-8b35-446e-90b8-51f3a59c6719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:47.973283 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dbbc06-2136-4bc8-896a-23652414f3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:47.974457 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbbc06-2136-4bc8-896a-23652414f3bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:47.974950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbbc06-2136-4bc8-896a-23652414f3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:47.975447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbbc06-2136-4bc8-896a-23652414f3bd admin admin] PATH_INFO: `/v3/projects/51e95123daab4cd285589cb523685623` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:47.982420 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dbbc06-2136-4bc8-896a-23652414f3bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=51e95123daab4cd285589cb523685623)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:47.986800 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dbbc06-2136-4bc8-896a-23652414f3bd admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:48.044591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c7895-8b35-446e-90b8-51f3a59c6719 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:48.057065 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251dee5d-db01-4b8d-927f-b789aaa10a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:48.059685 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251dee5d-db01-4b8d-927f-b789aaa10a50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:48.060200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251dee5d-db01-4b8d-927f-b789aaa10a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:48.060787 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251dee5d-db01-4b8d-927f-b789aaa10a50 admin admin] PATH_INFO: `/v3/projects/1d37787a60e74327ac4a19b4a6413f94/users/b7e29fa765434350980531b0d1aedbaa/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:48.123994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-81dbbc06-2136-4bc8-896a-23652414f3bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51e95123daab4cd285589cb523685623', 'request_id': 'req-81dbbc06-2136-4bc8-896a-23652414f3bd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:48.124993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251dee5d-db01-4b8d-927f-b789aaa10a50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d37787a60e74327ac4a19b4a6413f94, user_id=b7e29fa765434350980531b0d1aedbaa, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:48.131625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251dee5d-db01-4b8d-927f-b789aaa10a50 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:48.154315 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-81dbbc06-2136-4bc8-896a-23652414f3bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51e95123daab4cd285589cb523685623', 'request_id': 'req-81dbbc06-2136-4bc8-896a-23652414f3bd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:48.170895 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-81dbbc06-2136-4bc8-896a-23652414f3bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51e95123daab4cd285589cb523685623', 'request_id': 'req-81dbbc06-2136-4bc8-896a-23652414f3bd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:48.185551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:48.209975 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1027/2072] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:47 2022] PUT /identity/v3/projects/1d37787a60e74327ac4a19b4a6413f94/users/b7e29fa765434350980531b0d1aedbaa/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:48.231512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-81dbbc06-2136-4bc8-896a-23652414f3bd admin admin] The token cache is being invalidate because project 51e95123daab4cd285589cb523685623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:48.232399 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-81dbbc06-2136-4bc8-896a-23652414f3bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:48.234176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921f746-a899-472f-8db5-cbfb3b66540c None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:48.235262 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921f746-a899-472f-8db5-cbfb3b66540c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:48.235908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921f746-a899-472f-8db5-cbfb3b66540c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:48.237354 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-81dbbc06-2136-4bc8-896a-23652414f3bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:48.240164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1046/2073] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:14:47 2022] DELETE /identity/v3/projects/51e95123daab4cd285589cb523685623 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:48.252008 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfedd5a-fdbd-4c56-8a3c-6a09621e3298 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:48.252607 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfedd5a-fdbd-4c56-8a3c-6a09621e3298 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:48.254572 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfedd5a-fdbd-4c56-8a3c-6a09621e3298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:48.341465 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-5bfedd5a-fdbd-4c56-8a3c-6a09621e3298 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:48.341917 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-5921f746-a899-472f-8db5-cbfb3b66540c None None] MFA Rules not processed for user `b7e29fa765434350980531b0d1aedbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:48.344038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5bfedd5a-fdbd-4c56-8a3c-6a09621e3298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:48.344293 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-5921f746-a899-472f-8db5-cbfb3b66540c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:48.545614 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1028/2074] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:48.547207 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921f746-a899-472f-8db5-cbfb3b66540c None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:48.552755 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-5921f746-a899-472f-8db5-cbfb3b66540c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:48.641796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1047/2075] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:48.645785 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e706633-d48d-4ad8-af5d-44fe424432e9 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:48.646895 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e706633-d48d-4ad8-af5d-44fe424432e9 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:48.648585 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e706633-d48d-4ad8-af5d-44fe424432e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:48.673327 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:48.695589 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-0e706633-d48d-4ad8-af5d-44fe424432e9 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:48.698616 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0e706633-d48d-4ad8-af5d-44fe424432e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:48.722371 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1048/2076] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:48.727217 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e706633-d48d-4ad8-af5d-44fe424432e9 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:48.749936 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e706633-d48d-4ad8-af5d-44fe424432e9 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:48.759853 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae95f75-f2e6-4a91-a827-7769956418d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:48.761377 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae95f75-f2e6-4a91-a827-7769956418d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:48.761977 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae95f75-f2e6-4a91-a827-7769956418d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:48.762668 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae95f75-f2e6-4a91-a827-7769956418d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:48.763073 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921f746-a899-472f-8db5-cbfb3b66540c None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:48.764250 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae95f75-f2e6-4a91-a827-7769956418d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:48.764895 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae95f75-f2e6-4a91-a827-7769956418d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:48.768427 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae95f75-f2e6-4a91-a827-7769956418d5 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:48.774701 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4e0ac-8e62-472b-959c-59622af202b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:48.775984 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4e0ac-8e62-472b-959c-59622af202b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:48.776519 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4e0ac-8e62-472b-959c-59622af202b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:48.777010 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4e0ac-8e62-472b-959c-59622af202b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:48.779198 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4e0ac-8e62-472b-959c-59622af202b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:48.783478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4e0ac-8e62-472b-959c-59622af202b0 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:48.795351 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1049/2077] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:48.814078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae95f75-f2e6-4a91-a827-7769956418d5 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:48.863337 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae95f75-f2e6-4a91-a827-7769956418d5 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:48.887299 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c06dfa-e10f-4b1b-a109-73a2ae347553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:48.888539 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c06dfa-e10f-4b1b-a109-73a2ae347553 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:48.889180 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c06dfa-e10f-4b1b-a109-73a2ae347553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:48.889797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c06dfa-e10f-4b1b-a109-73a2ae347553 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:48.890684 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1029/2078] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:48 2022] POST /identity/v3/projects => generated 385 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:48.891372 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c06dfa-e10f-4b1b-a109-73a2ae347553 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:48.898990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c06dfa-e10f-4b1b-a109-73a2ae347553 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:48.899600 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4e0ac-8e62-472b-959c-59622af202b0 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:49.017026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1050/2079] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:48 2022] POST /identity/v3/projects => generated 373 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:49.018952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c06dfa-e10f-4b1b-a109-73a2ae347553 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:49.062406 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4e0ac-8e62-472b-959c-59622af202b0 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:49.278936 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c06dfa-e10f-4b1b-a109-73a2ae347553 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:49.288338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52454cc5-3c82-493e-9de1-02f141a3ff53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:49.289804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0ae12-65e8-441b-89a1-530eea37a2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:49.289999 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52454cc5-3c82-493e-9de1-02f141a3ff53 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:49.290220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52454cc5-3c82-493e-9de1-02f141a3ff53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:49.290518 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52454cc5-3c82-493e-9de1-02f141a3ff53 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:49.291154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52454cc5-3c82-493e-9de1-02f141a3ff53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:49.292256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0ae12-65e8-441b-89a1-530eea37a2fa admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:49.293759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52454cc5-3c82-493e-9de1-02f141a3ff53 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:49.294392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0ae12-65e8-441b-89a1-530eea37a2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:49.295137 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0ae12-65e8-441b-89a1-530eea37a2fa admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:49.297448 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0ae12-65e8-441b-89a1-530eea37a2fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:49.301812 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0ae12-65e8-441b-89a1-530eea37a2fa admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:49.345565 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1051/2080] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:49 2022] POST /identity/v3/users => generated 287 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:49.345565 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52454cc5-3c82-493e-9de1-02f141a3ff53 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:49.350342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-52454cc5-3c82-493e-9de1-02f141a3ff53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:49.374157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1030/2081] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:48 2022] POST /identity/v3/users => generated 299 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:49.375813 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0ae12-65e8-441b-89a1-530eea37a2fa admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:49.390104 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52454cc5-3c82-493e-9de1-02f141a3ff53 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:49.392661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0ae12-65e8-441b-89a1-530eea37a2fa admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:49.399903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195b193-b280-40a8-8ada-eda379b5f45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:49.401449 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195b193-b280-40a8-8ada-eda379b5f45b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:49.401966 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360dbcc-07f2-45b3-b49f-1a9e0ef116b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:49.402169 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195b193-b280-40a8-8ada-eda379b5f45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:49.402602 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360dbcc-07f2-45b3-b49f-1a9e0ef116b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:49.402796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195b193-b280-40a8-8ada-eda379b5f45b admin admin] PATH_INFO: `/v3/projects/bbf4b01c3cdd4b8f98406b488682f4aa` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:49.402985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360dbcc-07f2-45b3-b49f-1a9e0ef116b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:49.403154 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360dbcc-07f2-45b3-b49f-1a9e0ef116b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:49.404014 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360dbcc-07f2-45b3-b49f-1a9e0ef116b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:49.406265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360dbcc-07f2-45b3-b49f-1a9e0ef116b6 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:49.406864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195b193-b280-40a8-8ada-eda379b5f45b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbf4b01c3cdd4b8f98406b488682f4aa)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:49.411213 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195b193-b280-40a8-8ada-eda379b5f45b admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:49.424527 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1031/2082] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:49.424527 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360dbcc-07f2-45b3-b49f-1a9e0ef116b6 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:49.437256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360dbcc-07f2-45b3-b49f-1a9e0ef116b6 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:49.449540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca638b-e4d4-41c2-b005-b205ddfef9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:49.456338 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca638b-e4d4-41c2-b005-b205ddfef9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:49.456914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca638b-e4d4-41c2-b005-b205ddfef9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:49.457506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca638b-e4d4-41c2-b005-b205ddfef9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:49.458625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca638b-e4d4-41c2-b005-b205ddfef9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:49.461483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca638b-e4d4-41c2-b005-b205ddfef9a6 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:49.485160 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1032/2083] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:49.492776 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca638b-e4d4-41c2-b005-b205ddfef9a6 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:49.524979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca638b-e4d4-41c2-b005-b205ddfef9a6 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:49.537173 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:49.552869 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-1195b193-b280-40a8-8ada-eda379b5f45b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbf4b01c3cdd4b8f98406b488682f4aa', 'request_id': 'req-1195b193-b280-40a8-8ada-eda379b5f45b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:49.590058 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-1195b193-b280-40a8-8ada-eda379b5f45b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbf4b01c3cdd4b8f98406b488682f4aa', 'request_id': 'req-1195b193-b280-40a8-8ada-eda379b5f45b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:49.616275 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-1195b193-b280-40a8-8ada-eda379b5f45b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbf4b01c3cdd4b8f98406b488682f4aa', 'request_id': 'req-1195b193-b280-40a8-8ada-eda379b5f45b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:49.622958 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:49.659514 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-1195b193-b280-40a8-8ada-eda379b5f45b admin admin] The token cache is being invalidate because project bbf4b01c3cdd4b8f98406b488682f4aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:49.662155 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-1195b193-b280-40a8-8ada-eda379b5f45b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:49.666593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-1195b193-b280-40a8-8ada-eda379b5f45b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:49.669361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1052/2084] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:14:49 2022] DELETE /identity/v3/projects/bbf4b01c3cdd4b8f98406b488682f4aa => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:49.670342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195b193-b280-40a8-8ada-eda379b5f45b admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:49.672070 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-1195b193-b280-40a8-8ada-eda379b5f45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:49.719746 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6b599-6fb5-433a-a5eb-c65e3181c728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:49.721054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6b599-6fb5-433a-a5eb-c65e3181c728 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:49.721709 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6b599-6fb5-433a-a5eb-c65e3181c728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:49.722736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6b599-6fb5-433a-a5eb-c65e3181c728 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:49.724899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6b599-6fb5-433a-a5eb-c65e3181c728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:49.728307 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6b599-6fb5-433a-a5eb-c65e3181c728 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:49.771101 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1033/2085] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:49.771101 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6b599-6fb5-433a-a5eb-c65e3181c728 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:49.780877 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-71c6b599-6fb5-433a-a5eb-c65e3181c728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:49.831370 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6b599-6fb5-433a-a5eb-c65e3181c728 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:49.843259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553a2d4-d218-4218-aa54-873bbf6c5c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:49.844644 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553a2d4-d218-4218-aa54-873bbf6c5c15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:49.845297 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553a2d4-d218-4218-aa54-873bbf6c5c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:49.845918 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553a2d4-d218-4218-aa54-873bbf6c5c15 admin admin] PATH_INFO: `/v3/users/14360e05265c4ec580123dc9946ba6a8` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:49.848913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553a2d4-d218-4218-aa54-873bbf6c5c15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14360e05265c4ec580123dc9946ba6a8)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:49.855423 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553a2d4-d218-4218-aa54-873bbf6c5c15 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:49.911572 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195b193-b280-40a8-8ada-eda379b5f45b admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:49.921437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cb2e49-e09c-4bde-9bcf-ff4090e304d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:49.922528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb2e49-e09c-4bde-9bcf-ff4090e304d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:49.922808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb2e49-e09c-4bde-9bcf-ff4090e304d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:49.923383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb2e49-e09c-4bde-9bcf-ff4090e304d5 admin admin] PATH_INFO: `/v3/projects/ad0b809524224af2ac8bd128589e9ff1/users/bbc20ae691c3461d9a8741c0f4881d51/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:49.949765 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cb2e49-e09c-4bde-9bcf-ff4090e304d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad0b809524224af2ac8bd128589e9ff1, user_id=bbc20ae691c3461d9a8741c0f4881d51, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:49.953284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cb2e49-e09c-4bde-9bcf-ff4090e304d5 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:49.976845 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1053/2086] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:49 2022] PUT /identity/v3/projects/ad0b809524224af2ac8bd128589e9ff1/users/bbc20ae691c3461d9a8741c0f4881d51/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:49.978247 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e23a7-e4ce-440a-9509-4c00a1812151 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:49.978553 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e23a7-e4ce-440a-9509-4c00a1812151 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:49.978959 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e23a7-e4ce-440a-9509-4c00a1812151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:49.991044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0553a2d4-d218-4218-aa54-873bbf6c5c15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14360e05265c4ec580123dc9946ba6a8'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:49.992505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0553a2d4-d218-4218-aa54-873bbf6c5c15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14360e05265c4ec580123dc9946ba6a8'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:50.006002 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-a57e23a7-e4ce-440a-9509-4c00a1812151 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:50.008046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a57e23a7-e4ce-440a-9509-4c00a1812151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:50.015513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0553a2d4-d218-4218-aa54-873bbf6c5c15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14360e05265c4ec580123dc9946ba6a8'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:50.016434 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0553a2d4-d218-4218-aa54-873bbf6c5c15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14360e05265c4ec580123dc9946ba6a8'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:50.045183 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0553a2d4-d218-4218-aa54-873bbf6c5c15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14360e05265c4ec580123dc9946ba6a8'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:50.080738 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1034/2087] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:14:49 2022] DELETE /identity/v3/users/14360e05265c4ec580123dc9946ba6a8 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:50.080738 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553a2d4-d218-4218-aa54-873bbf6c5c15 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:50.082554 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-0553a2d4-d218-4218-aa54-873bbf6c5c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:50.117600 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1054/2088] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:50.119101 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e23a7-e4ce-440a-9509-4c00a1812151 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:50.120866 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a57e23a7-e4ce-440a-9509-4c00a1812151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:50.145748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e23a7-e4ce-440a-9509-4c00a1812151 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:50.231767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553a2d4-d218-4218-aa54-873bbf6c5c15 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:50.239651 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47e953-91cf-40e5-8130-bc958b7f374c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:50.240396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47e953-91cf-40e5-8130-bc958b7f374c admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:50.240739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47e953-91cf-40e5-8130-bc958b7f374c admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:50.241018 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47e953-91cf-40e5-8130-bc958b7f374c admin admin] PATH_INFO: `/v3/projects/5c7fb0f42d4d413ba366897936e129fd/users/eb60592f0e034007ae0561113310fb4a/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:50.282125 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de47e953-91cf-40e5-8130-bc958b7f374c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c7fb0f42d4d413ba366897936e129fd, user_id=eb60592f0e034007ae0561113310fb4a, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:50.287180 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de47e953-91cf-40e5-8130-bc958b7f374c admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:50.290496 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:50.317618 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1035/2089] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:50 2022] PUT /identity/v3/projects/5c7fb0f42d4d413ba366897936e129fd/users/eb60592f0e034007ae0561113310fb4a/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:50.321181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f84f2-4fcd-47c6-a9bb-86e645d42ad5 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:50.321709 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f84f2-4fcd-47c6-a9bb-86e645d42ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:50.322160 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f84f2-4fcd-47c6-a9bb-86e645d42ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:50.325190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0f9ae-c1df-4204-9a7b-5d1671e4ff3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:50.326394 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0f9ae-c1df-4204-9a7b-5d1671e4ff3c admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:50.327549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0f9ae-c1df-4204-9a7b-5d1671e4ff3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:50.328191 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0f9ae-c1df-4204-9a7b-5d1671e4ff3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:50.329309 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0f9ae-c1df-4204-9a7b-5d1671e4ff3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:50.331446 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0f9ae-c1df-4204-9a7b-5d1671e4ff3c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:50.349490 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1055/2090] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:50.351893 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6dbc9-bae2-4950-bc31-252c01beae16 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:50.352385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6dbc9-bae2-4950-bc31-252c01beae16 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:50.353258 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6dbc9-bae2-4950-bc31-252c01beae16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:50.374541 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-ed4f84f2-4fcd-47c6-a9bb-86e645d42ad5 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:50.377052 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ed4f84f2-4fcd-47c6-a9bb-86e645d42ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:50.386402 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-14a6dbc9-bae2-4950-bc31-252c01beae16 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:50.389564 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-14a6dbc9-bae2-4950-bc31-252c01beae16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:50.409320 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:50.521460 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1056/2091] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:50.523939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63325a9f-68f2-435d-a433-2e235c4e8ca9 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:50.524481 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63325a9f-68f2-435d-a433-2e235c4e8ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:50.524948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63325a9f-68f2-435d-a433-2e235c4e8ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:50.545682 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1036/2092] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:50.552578 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f84f2-4fcd-47c6-a9bb-86e645d42ad5 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:50.569130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f84f2-4fcd-47c6-a9bb-86e645d42ad5 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:50.575128 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-63325a9f-68f2-435d-a433-2e235c4e8ca9 None None] MFA Rules not processed for user `eb60592f0e034007ae0561113310fb4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:50.588184 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a8fb6-d41a-4b46-a11d-cc46438e7f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:50.588184 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a8fb6-d41a-4b46-a11d-cc46438e7f69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:50.588184 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a8fb6-d41a-4b46-a11d-cc46438e7f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:50.588184 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a8fb6-d41a-4b46-a11d-cc46438e7f69 admin admin] PATH_INFO: `/v3/projects/ad0b809524224af2ac8bd128589e9ff1/users/bbc20ae691c3461d9a8741c0f4881d51/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:50.593017 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-63325a9f-68f2-435d-a433-2e235c4e8ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:50.597664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a8fb6-d41a-4b46-a11d-cc46438e7f69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad0b809524224af2ac8bd128589e9ff1, user_id=bbc20ae691c3461d9a8741c0f4881d51, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:50.603930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a8fb6-d41a-4b46-a11d-cc46438e7f69 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:50.622230 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1037/2093] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:50 2022] PUT /identity/v3/projects/ad0b809524224af2ac8bd128589e9ff1/users/bbc20ae691c3461d9a8741c0f4881d51/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:50.625023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a8fb6-d41a-4b46-a11d-cc46438e7f69 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:50.768618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1057/2094] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:50.768618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63325a9f-68f2-435d-a433-2e235c4e8ca9 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:50.768618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-63325a9f-68f2-435d-a433-2e235c4e8ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:50.842546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a8fb6-d41a-4b46-a11d-cc46438e7f69 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:50.864002 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c3621-b6a4-4c3c-abcc-fdb0907fefde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:50.864986 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c3621-b6a4-4c3c-abcc-fdb0907fefde admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:50.865442 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c3621-b6a4-4c3c-abcc-fdb0907fefde admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:50.865848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c3621-b6a4-4c3c-abcc-fdb0907fefde admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:50.866872 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c3621-b6a4-4c3c-abcc-fdb0907fefde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:50.867344 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c3621-b6a4-4c3c-abcc-fdb0907fefde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:50.869857 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c3621-b6a4-4c3c-abcc-fdb0907fefde admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:50.899884 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63325a9f-68f2-435d-a433-2e235c4e8ca9 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:50.912627 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1038/2095] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:50.914731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8153e-9d4b-4702-b051-f101747540b3 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:50.915091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8153e-9d4b-4702-b051-f101747540b3 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:50.915658 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8153e-9d4b-4702-b051-f101747540b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:50.964721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-31f8153e-9d4b-4702-b051-f101747540b3 None None] MFA Rules not processed for user `bbc20ae691c3461d9a8741c0f4881d51`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:50.967249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-31f8153e-9d4b-4702-b051-f101747540b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:50.981718 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:51.078138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e0887-28c1-4b8a-906c-57086f8ee49e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:51.082583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e0887-28c1-4b8a-906c-57086f8ee49e service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:51.083071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e0887-28c1-4b8a-906c-57086f8ee49e service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:51.083542 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e0887-28c1-4b8a-906c-57086f8ee49e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:51.156122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1039/2096] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:51.158362 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8153e-9d4b-4702-b051-f101747540b3 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:51.161291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e0887-28c1-4b8a-906c-57086f8ee49e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:51.161612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-31f8153e-9d4b-4702-b051-f101747540b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:51.170536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e0887-28c1-4b8a-906c-57086f8ee49e service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:51.236374 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:51.264409 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1058/2097] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:51.267176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e0887-28c1-4b8a-906c-57086f8ee49e service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:51.331642 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8153e-9d4b-4702-b051-f101747540b3 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:51.345433 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e584a-4815-426d-983b-6102c733283f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:51.350518 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e584a-4815-426d-983b-6102c733283f service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:51.350518 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e584a-4815-426d-983b-6102c733283f service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:51.350518 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e584a-4815-426d-983b-6102c733283f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:51.359091 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e0887-28c1-4b8a-906c-57086f8ee49e service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:51.401338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab58b2-746c-40b8-9280-aee617388f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:51.402024 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab58b2-746c-40b8-9280-aee617388f0a admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:51.402300 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab58b2-746c-40b8-9280-aee617388f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:51.402554 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab58b2-746c-40b8-9280-aee617388f0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:51.403254 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab58b2-746c-40b8-9280-aee617388f0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:51.405604 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab58b2-746c-40b8-9280-aee617388f0a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:51.518040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1059/2098] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:51 2022] POST /identity/v3/projects => generated 385 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:51.518040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab58b2-746c-40b8-9280-aee617388f0a admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:51.518040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-30ab58b2-746c-40b8-9280-aee617388f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:51.599662 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e584a-4815-426d-983b-6102c733283f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:51.606156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e584a-4815-426d-983b-6102c733283f service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:51.640385 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1040/2099] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:51.653748 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e584a-4815-426d-983b-6102c733283f service nova] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:51.664163 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab58b2-746c-40b8-9280-aee617388f0a admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:51.671134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a9f99-b886-4ee5-85b1-acf50eacc327 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:51.672832 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a9f99-b886-4ee5-85b1-acf50eacc327 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:51.673361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a9f99-b886-4ee5-85b1-acf50eacc327 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:51.673880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a9f99-b886-4ee5-85b1-acf50eacc327 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:51.678694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-142a9f99-b886-4ee5-85b1-acf50eacc327 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:51.715158 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e584a-4815-426d-983b-6102c733283f service nova] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:51.885017 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a9f99-b886-4ee5-85b1-acf50eacc327 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:51.886561 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a9f99-b886-4ee5-85b1-acf50eacc327 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:51.909991 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1060/2100] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:51.942202 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c80ffc-0f5b-49d3-8fa8-c7bb4cfce3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:51.943056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c80ffc-0f5b-49d3-8fa8-c7bb4cfce3cb admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:51.943204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c80ffc-0f5b-49d3-8fa8-c7bb4cfce3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:51.943401 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c80ffc-0f5b-49d3-8fa8-c7bb4cfce3cb admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:51.946702 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c80ffc-0f5b-49d3-8fa8-c7bb4cfce3cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:51.955365 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c80ffc-0f5b-49d3-8fa8-c7bb4cfce3cb admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:52.009258 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a9f99-b886-4ee5-85b1-acf50eacc327 service cinder] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:52.016716 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1041/2101] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:51 2022] POST /identity/v3/users => generated 293 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:52.026542 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a9f99-b886-4ee5-85b1-acf50eacc327 service cinder] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:52.042580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aadc57-c232-4725-a66d-5621d15307f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:52.043469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aadc57-c232-4725-a66d-5621d15307f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:52.043778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aadc57-c232-4725-a66d-5621d15307f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:52.044276 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aadc57-c232-4725-a66d-5621d15307f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:52.045072 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aadc57-c232-4725-a66d-5621d15307f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:52.046925 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aadc57-c232-4725-a66d-5621d15307f0 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:52.086639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1061/2102] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:52.090783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c80ffc-0f5b-49d3-8fa8-c7bb4cfce3cb admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:52.113630 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c80ffc-0f5b-49d3-8fa8-c7bb4cfce3cb admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:52.123290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53659245-b138-4bd7-92b8-22eddbdf079a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:52.124737 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53659245-b138-4bd7-92b8-22eddbdf079a admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:52.125200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53659245-b138-4bd7-92b8-22eddbdf079a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:52.130028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53659245-b138-4bd7-92b8-22eddbdf079a admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:52.131458 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53659245-b138-4bd7-92b8-22eddbdf079a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:52.138071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53659245-b138-4bd7-92b8-22eddbdf079a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:52.183164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aadc57-c232-4725-a66d-5621d15307f0 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:52.197398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1042/2103] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:52.204598 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53659245-b138-4bd7-92b8-22eddbdf079a admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:52.211667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aadc57-c232-4725-a66d-5621d15307f0 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:52.231377 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53659245-b138-4bd7-92b8-22eddbdf079a admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:52.239099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c76da-2ec7-4578-a7e5-6ec71dca4227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:52.240105 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c76da-2ec7-4578-a7e5-6ec71dca4227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:52.240451 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c76da-2ec7-4578-a7e5-6ec71dca4227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:52.240758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c76da-2ec7-4578-a7e5-6ec71dca4227 admin admin] PATH_INFO: `/v3/users/eb60592f0e034007ae0561113310fb4a` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:52.243543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c76da-2ec7-4578-a7e5-6ec71dca4227 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb60592f0e034007ae0561113310fb4a)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:52.248192 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c76da-2ec7-4578-a7e5-6ec71dca4227 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:52.248695 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4275f-feb0-40c4-9d7e-d45be6584f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:52.251354 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4275f-feb0-40c4-9d7e-d45be6584f3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:52.252237 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4275f-feb0-40c4-9d7e-d45be6584f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:52.252973 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4275f-feb0-40c4-9d7e-d45be6584f3f admin admin] PATH_INFO: `/v3/projects/1ff44d81ce674d92ab0dca9a4d0f3ee8/users/2409114e7514497e99f44f5b5d320c26/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:52.304120 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4275f-feb0-40c4-9d7e-d45be6584f3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ff44d81ce674d92ab0dca9a4d0f3ee8, user_id=2409114e7514497e99f44f5b5d320c26, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:52.316451 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4275f-feb0-40c4-9d7e-d45be6584f3f admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:52.368164 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1043/2104] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:52 2022] PUT /identity/v3/projects/1ff44d81ce674d92ab0dca9a4d0f3ee8/users/2409114e7514497e99f44f5b5d320c26/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:52.384371 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d41db-ced9-4989-a491-631846bbfe9a None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:52.384932 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d41db-ced9-4989-a491-631846bbfe9a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:52.388722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d41db-ced9-4989-a491-631846bbfe9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:52.414403 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ff1c76da-2ec7-4578-a7e5-6ec71dca4227 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb60592f0e034007ae0561113310fb4a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:52.434098 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:52.470445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ff1c76da-2ec7-4578-a7e5-6ec71dca4227 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb60592f0e034007ae0561113310fb4a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:52.506577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ff1c76da-2ec7-4578-a7e5-6ec71dca4227 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb60592f0e034007ae0561113310fb4a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:52.516103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ff1c76da-2ec7-4578-a7e5-6ec71dca4227 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb60592f0e034007ae0561113310fb4a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:52.516326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-8b1d41db-ced9-4989-a491-631846bbfe9a None None] MFA Rules not processed for user `2409114e7514497e99f44f5b5d320c26`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:52.519441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ff1c76da-2ec7-4578-a7e5-6ec71dca4227 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb60592f0e034007ae0561113310fb4a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:52.532932 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-8b1d41db-ced9-4989-a491-631846bbfe9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:52.571449 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:52.620037 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1062/2105] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:14:52 2022] DELETE /identity/v3/users/eb60592f0e034007ae0561113310fb4a => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:52.622139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c76da-2ec7-4578-a7e5-6ec71dca4227 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:52.624640 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ff1c76da-2ec7-4578-a7e5-6ec71dca4227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:52.717700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1044/2106] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:52.721066 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd9d7c-0dbc-4b3d-99d8-d531e7662369 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:52.722258 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd9d7c-0dbc-4b3d-99d8-d531e7662369 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:52.723161 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd9d7c-0dbc-4b3d-99d8-d531e7662369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:52.782829 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-e4bd9d7c-0dbc-4b3d-99d8-d531e7662369 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:52.786306 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e4bd9d7c-0dbc-4b3d-99d8-d531e7662369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:52.823955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c76da-2ec7-4578-a7e5-6ec71dca4227 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:52.833342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f949493-1b79-459a-a3a0-dba31c6594ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:52.835030 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f949493-1b79-459a-a3a0-dba31c6594ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:52.836043 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f949493-1b79-459a-a3a0-dba31c6594ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:52.836874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f949493-1b79-459a-a3a0-dba31c6594ea admin admin] PATH_INFO: `/v3/projects/5056a3ecae834c8fbb3baea7c1e67c14` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:52.839631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f949493-1b79-459a-a3a0-dba31c6594ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=5056a3ecae834c8fbb3baea7c1e67c14)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:52.844494 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f949493-1b79-459a-a3a0-dba31c6594ea admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:53.061055 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1045/2107] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:53.064698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd9d7c-0dbc-4b3d-99d8-d531e7662369 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:53.070999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e4bd9d7c-0dbc-4b3d-99d8-d531e7662369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:53.083385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-1f949493-1b79-459a-a3a0-dba31c6594ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5056a3ecae834c8fbb3baea7c1e67c14', 'request_id': 'req-1f949493-1b79-459a-a3a0-dba31c6594ea'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:53.108257 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-1f949493-1b79-459a-a3a0-dba31c6594ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5056a3ecae834c8fbb3baea7c1e67c14', 'request_id': 'req-1f949493-1b79-459a-a3a0-dba31c6594ea'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:53.130463 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-1f949493-1b79-459a-a3a0-dba31c6594ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5056a3ecae834c8fbb3baea7c1e67c14', 'request_id': 'req-1f949493-1b79-459a-a3a0-dba31c6594ea'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:53.155859 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-1f949493-1b79-459a-a3a0-dba31c6594ea admin admin] The token cache is being invalidate because project 5056a3ecae834c8fbb3baea7c1e67c14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:53.156339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-1f949493-1b79-459a-a3a0-dba31c6594ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:53.157528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-1f949493-1b79-459a-a3a0-dba31c6594ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:53.166540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1063/2108] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:14:52 2022] DELETE /identity/v3/projects/5056a3ecae834c8fbb3baea7c1e67c14 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:53.168157 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f949493-1b79-459a-a3a0-dba31c6594ea admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:53.170506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-1f949493-1b79-459a-a3a0-dba31c6594ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:53.206071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd9d7c-0dbc-4b3d-99d8-d531e7662369 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:53.316646 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e9d1a8-4724-4ee7-a945-a32f6686d21b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:53.317662 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9d1a8-4724-4ee7-a945-a32f6686d21b service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:53.318113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9d1a8-4724-4ee7-a945-a32f6686d21b service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:53.318589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9d1a8-4724-4ee7-a945-a32f6686d21b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:53.322099 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c6e9d1a8-4724-4ee7-a945-a32f6686d21b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:53.362442 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:53.393369 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f949493-1b79-459a-a3a0-dba31c6594ea admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:53.402280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae77bb-6443-4e25-b680-e87e82138cd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:53.403222 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae77bb-6443-4e25-b680-e87e82138cd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:53.403808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae77bb-6443-4e25-b680-e87e82138cd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:53.404121 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae77bb-6443-4e25-b680-e87e82138cd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:53.406941 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-d5ae77bb-6443-4e25-b680-e87e82138cd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:53.436788 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:53.461031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e9d1a8-4724-4ee7-a945-a32f6686d21b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:53.464108 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e9d1a8-4724-4ee7-a945-a32f6686d21b service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:53.483992 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1046/2109] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:53.485568 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e9d1a8-4724-4ee7-a945-a32f6686d21b service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:53.488380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c6e9d1a8-4724-4ee7-a945-a32f6686d21b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:53.622677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ae77bb-6443-4e25-b680-e87e82138cd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:53.624736 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ae77bb-6443-4e25-b680-e87e82138cd7 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:53.644536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1064/2110] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:53.646237 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e9d1a8-4724-4ee7-a945-a32f6686d21b service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:53.650920 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae77bb-6443-4e25-b680-e87e82138cd7 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:53.655177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77913a-1a3e-4f92-8a18-88321f802680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:53.656062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77913a-1a3e-4f92-8a18-88321f802680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:53.656556 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77913a-1a3e-4f92-8a18-88321f802680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:53.657165 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77913a-1a3e-4f92-8a18-88321f802680 admin admin] PATH_INFO: `/v3/users/e181d37ce3df4097abd7b5d59a9651f7` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:53.659636 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77913a-1a3e-4f92-8a18-88321f802680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e181d37ce3df4097abd7b5d59a9651f7)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:53.661786 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77913a-1a3e-4f92-8a18-88321f802680 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:53.662403 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-d5ae77bb-6443-4e25-b680-e87e82138cd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:53.686458 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae77bb-6443-4e25-b680-e87e82138cd7 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:53.693860 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566936f-6d33-412b-99ff-f79699c8ff25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:53.695104 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566936f-6d33-412b-99ff-f79699c8ff25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:53.695595 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566936f-6d33-412b-99ff-f79699c8ff25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:53.695886 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566936f-6d33-412b-99ff-f79699c8ff25 admin admin] PATH_INFO: `/v3/users/2409114e7514497e99f44f5b5d320c26` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:53.700840 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566936f-6d33-412b-99ff-f79699c8ff25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2409114e7514497e99f44f5b5d320c26)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:53.703712 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566936f-6d33-412b-99ff-f79699c8ff25 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:53.819093 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a566936f-6d33-412b-99ff-f79699c8ff25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2409114e7514497e99f44f5b5d320c26'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:53.820084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0f77913a-1a3e-4f92-8a18-88321f802680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e181d37ce3df4097abd7b5d59a9651f7'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:53.821593 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0f77913a-1a3e-4f92-8a18-88321f802680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e181d37ce3df4097abd7b5d59a9651f7'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:53.831611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a566936f-6d33-412b-99ff-f79699c8ff25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2409114e7514497e99f44f5b5d320c26'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:53.843026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0f77913a-1a3e-4f92-8a18-88321f802680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e181d37ce3df4097abd7b5d59a9651f7'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:53.848129 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0f77913a-1a3e-4f92-8a18-88321f802680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e181d37ce3df4097abd7b5d59a9651f7'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:53.848613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a566936f-6d33-412b-99ff-f79699c8ff25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2409114e7514497e99f44f5b5d320c26'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:53.849471 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a566936f-6d33-412b-99ff-f79699c8ff25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2409114e7514497e99f44f5b5d320c26'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:53.851954 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a566936f-6d33-412b-99ff-f79699c8ff25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2409114e7514497e99f44f5b5d320c26'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:53.889447 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0f77913a-1a3e-4f92-8a18-88321f802680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e181d37ce3df4097abd7b5d59a9651f7'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:53.896070 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1065/2111] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:14:53 2022] DELETE /identity/v3/users/2409114e7514497e99f44f5b5d320c26 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:53.907839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6549b-8ae1-4a3e-b764-d0bdee03200d None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:53.908737 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6549b-8ae1-4a3e-b764-d0bdee03200d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:53.909257 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6549b-8ae1-4a3e-b764-d0bdee03200d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:53.952865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1047/2112] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:14:53 2022] DELETE /identity/v3/users/e181d37ce3df4097abd7b5d59a9651f7 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:53.971875 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-d1a6549b-8ae1-4a3e-b764-d0bdee03200d None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:53.975907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-d1a6549b-8ae1-4a3e-b764-d0bdee03200d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:54.266764 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1066/2113] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:54.288039 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77913a-1a3e-4f92-8a18-88321f802680 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:54.291023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-0f77913a-1a3e-4f92-8a18-88321f802680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:54.454313 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:54.511178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77913a-1a3e-4f92-8a18-88321f802680 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:54.545522 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fdb030-fb56-4770-9085-da84b27f6e23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:54.547754 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fdb030-fb56-4770-9085-da84b27f6e23 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:54.548374 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fdb030-fb56-4770-9085-da84b27f6e23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:54.549021 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fdb030-fb56-4770-9085-da84b27f6e23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:54.587128 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fdb030-fb56-4770-9085-da84b27f6e23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:54.592520 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fdb030-fb56-4770-9085-da84b27f6e23 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:54.614198 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6549b-8ae1-4a3e-b764-d0bdee03200d None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:54.615775 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-d1a6549b-8ae1-4a3e-b764-d0bdee03200d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:54.621277 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1048/2114] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:54.642777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6549b-8ae1-4a3e-b764-d0bdee03200d None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:54.655162 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e34ee3-457c-4c28-b712-0c27d2fd5fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:54.656139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e34ee3-457c-4c28-b712-0c27d2fd5fcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:54.656419 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e34ee3-457c-4c28-b712-0c27d2fd5fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:54.656729 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e34ee3-457c-4c28-b712-0c27d2fd5fcd admin admin] PATH_INFO: `/v3/projects/ad39e84f611a4f13bb145c11663e9f76` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:54.662265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e34ee3-457c-4c28-b712-0c27d2fd5fcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad39e84f611a4f13bb145c11663e9f76)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:54.664704 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e34ee3-457c-4c28-b712-0c27d2fd5fcd admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:54.689548 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fdb030-fb56-4770-9085-da84b27f6e23 service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:54.694873 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a4fdb030-fb56-4770-9085-da84b27f6e23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:54.714514 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:54.830645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-30e34ee3-457c-4c28-b712-0c27d2fd5fcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad39e84f611a4f13bb145c11663e9f76', 'request_id': 'req-30e34ee3-457c-4c28-b712-0c27d2fd5fcd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:54.860956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-30e34ee3-457c-4c28-b712-0c27d2fd5fcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad39e84f611a4f13bb145c11663e9f76', 'request_id': 'req-30e34ee3-457c-4c28-b712-0c27d2fd5fcd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:54.888868 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-30e34ee3-457c-4c28-b712-0c27d2fd5fcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad39e84f611a4f13bb145c11663e9f76', 'request_id': 'req-30e34ee3-457c-4c28-b712-0c27d2fd5fcd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:54.906020 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fdb030-fb56-4770-9085-da84b27f6e23 service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:54.925511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-30e34ee3-457c-4c28-b712-0c27d2fd5fcd admin admin] The token cache is being invalidate because project ad39e84f611a4f13bb145c11663e9f76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:54.926311 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-30e34ee3-457c-4c28-b712-0c27d2fd5fcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:54.928502 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-30e34ee3-457c-4c28-b712-0c27d2fd5fcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:54.933276 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1067/2115] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:14:54 2022] DELETE /identity/v3/projects/ad39e84f611a4f13bb145c11663e9f76 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:14:54.935472 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e34ee3-457c-4c28-b712-0c27d2fd5fcd admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:54.938944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-30e34ee3-457c-4c28-b712-0c27d2fd5fcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:55.072952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd297c90-938e-405f-be3e-697ed6f739aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:55.074540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd297c90-938e-405f-be3e-697ed6f739aa service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:55.075212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd297c90-938e-405f-be3e-697ed6f739aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:55.075706 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd297c90-938e-405f-be3e-697ed6f739aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:55.079313 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-dd297c90-938e-405f-be3e-697ed6f739aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:55.163941 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e34ee3-457c-4c28-b712-0c27d2fd5fcd admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:55.173206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba6ef8-83ff-4860-b10c-846ed36066fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:55.174543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba6ef8-83ff-4860-b10c-846ed36066fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:55.175191 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba6ef8-83ff-4860-b10c-846ed36066fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:55.175835 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba6ef8-83ff-4860-b10c-846ed36066fd admin admin] PATH_INFO: `/v3/projects/5c7fb0f42d4d413ba366897936e129fd` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:55.178515 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba6ef8-83ff-4860-b10c-846ed36066fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c7fb0f42d4d413ba366897936e129fd)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:55.182923 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba6ef8-83ff-4860-b10c-846ed36066fd admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:55.274749 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-1bba6ef8-83ff-4860-b10c-846ed36066fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c7fb0f42d4d413ba366897936e129fd', 'request_id': 'req-1bba6ef8-83ff-4860-b10c-846ed36066fd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:55.293314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-1bba6ef8-83ff-4860-b10c-846ed36066fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c7fb0f42d4d413ba366897936e129fd', 'request_id': 'req-1bba6ef8-83ff-4860-b10c-846ed36066fd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:55.312051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd297c90-938e-405f-be3e-697ed6f739aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:55.316599 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd297c90-938e-405f-be3e-697ed6f739aa service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:55.316890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-1bba6ef8-83ff-4860-b10c-846ed36066fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c7fb0f42d4d413ba366897936e129fd', 'request_id': 'req-1bba6ef8-83ff-4860-b10c-846ed36066fd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:55.353882 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-1bba6ef8-83ff-4860-b10c-846ed36066fd admin admin] The token cache is being invalidate because project 5c7fb0f42d4d413ba366897936e129fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:55.354635 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-1bba6ef8-83ff-4860-b10c-846ed36066fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:55.356174 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-1bba6ef8-83ff-4860-b10c-846ed36066fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:55.365232 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1068/2116] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:14:54 2022] DELETE /identity/v3/projects/5c7fb0f42d4d413ba366897936e129fd => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:55.365546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba6ef8-83ff-4860-b10c-846ed36066fd admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:55.368825 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-1bba6ef8-83ff-4860-b10c-846ed36066fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:55.372604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1049/2117] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:55.381009 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf64914-8719-4ee3-9f12-204ed74fc714 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:55.383693 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf64914-8719-4ee3-9f12-204ed74fc714 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:55.385089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf64914-8719-4ee3-9f12-204ed74fc714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:55.467495 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-fbf64914-8719-4ee3-9f12-204ed74fc714 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:55.474175 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-fbf64914-8719-4ee3-9f12-204ed74fc714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:55.532246 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:55.606158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba6ef8-83ff-4860-b10c-846ed36066fd admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:55.622295 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe54e59-9988-4850-9943-15d915084619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:55.623374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe54e59-9988-4850-9943-15d915084619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:55.624149 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe54e59-9988-4850-9943-15d915084619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:55.624971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe54e59-9988-4850-9943-15d915084619 admin admin] PATH_INFO: `/v3/users/c629935f44cf4d6fb51c57d118b61d14` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:55.626946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe54e59-9988-4850-9943-15d915084619 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c629935f44cf4d6fb51c57d118b61d14)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:55.629504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe54e59-9988-4850-9943-15d915084619 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:55.645849 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:55.684427 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1050/2118] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:55.684427 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf64914-8719-4ee3-9f12-204ed74fc714 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:55.686612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-fbf64914-8719-4ee3-9f12-204ed74fc714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:55.708266 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf64914-8719-4ee3-9f12-204ed74fc714 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:55.715523 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8046712-9eec-4c4f-849b-cae10dbc59d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:55.716204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8046712-9eec-4c4f-849b-cae10dbc59d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:55.716526 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8046712-9eec-4c4f-849b-cae10dbc59d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:55.716809 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8046712-9eec-4c4f-849b-cae10dbc59d5 admin admin] PATH_INFO: `/v3/projects/1ff44d81ce674d92ab0dca9a4d0f3ee8` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:55.718157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8046712-9eec-4c4f-849b-cae10dbc59d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ff44d81ce674d92ab0dca9a4d0f3ee8)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:55.720323 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8046712-9eec-4c4f-849b-cae10dbc59d5 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:55.774370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5fe54e59-9988-4850-9943-15d915084619 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c629935f44cf4d6fb51c57d118b61d14'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:55.795869 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5fe54e59-9988-4850-9943-15d915084619 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c629935f44cf4d6fb51c57d118b61d14'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:55.805809 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c8046712-9eec-4c4f-849b-cae10dbc59d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ff44d81ce674d92ab0dca9a4d0f3ee8', 'request_id': 'req-c8046712-9eec-4c4f-849b-cae10dbc59d5'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:55.822078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5fe54e59-9988-4850-9943-15d915084619 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c629935f44cf4d6fb51c57d118b61d14'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:55.823238 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5fe54e59-9988-4850-9943-15d915084619 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c629935f44cf4d6fb51c57d118b61d14'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:55.824126 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5fe54e59-9988-4850-9943-15d915084619 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c629935f44cf4d6fb51c57d118b61d14'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:55.831520 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c8046712-9eec-4c4f-849b-cae10dbc59d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ff44d81ce674d92ab0dca9a4d0f3ee8', 'request_id': 'req-c8046712-9eec-4c4f-849b-cae10dbc59d5'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:55.845450 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c8046712-9eec-4c4f-849b-cae10dbc59d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ff44d81ce674d92ab0dca9a4d0f3ee8', 'request_id': 'req-c8046712-9eec-4c4f-849b-cae10dbc59d5'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:55.872912 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1069/2119] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:14:55 2022] DELETE /identity/v3/users/c629935f44cf4d6fb51c57d118b61d14 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:55.880308 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca700be-bc29-4e8f-8ae3-fb061bcce1db None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:55.880557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c8046712-9eec-4c4f-849b-cae10dbc59d5 admin admin] The token cache is being invalidate because project 1ff44d81ce674d92ab0dca9a4d0f3ee8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:55.880855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca700be-bc29-4e8f-8ae3-fb061bcce1db None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:55.881031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c8046712-9eec-4c4f-849b-cae10dbc59d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:55.881298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca700be-bc29-4e8f-8ae3-fb061bcce1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:55.883346 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c8046712-9eec-4c4f-849b-cae10dbc59d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:55.892276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1051/2120] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:14:55 2022] DELETE /identity/v3/projects/1ff44d81ce674d92ab0dca9a4d0f3ee8 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:55.895115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3ce63-b889-4ede-9261-6bf1a87d99bd None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:55.895590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3ce63-b889-4ede-9261-6bf1a87d99bd None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:55.896398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3ce63-b889-4ede-9261-6bf1a87d99bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:55.942217 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-8ca700be-bc29-4e8f-8ae3-fb061bcce1db None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:55.949668 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-8ca700be-bc29-4e8f-8ae3-fb061bcce1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:55.949882 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-a2e3ce63-b889-4ede-9261-6bf1a87d99bd None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:55.952729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a2e3ce63-b889-4ede-9261-6bf1a87d99bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:56.105471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1052/2121] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:56.137985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5137216-f1ee-4cd2-bbf6-c43f1ebde61d None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:56.138534 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5137216-f1ee-4cd2-bbf6-c43f1ebde61d None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:56.139187 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5137216-f1ee-4cd2-bbf6-c43f1ebde61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:56.180478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-e5137216-f1ee-4cd2-bbf6-c43f1ebde61d None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:56.182043 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e5137216-f1ee-4cd2-bbf6-c43f1ebde61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:56.194702 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1053/2122] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:56.211789 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5137216-f1ee-4cd2-bbf6-c43f1ebde61d None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:56.237749 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5137216-f1ee-4cd2-bbf6-c43f1ebde61d None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:56.246368 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac30f3-5974-4465-91ad-fe10eafa0c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:56.247185 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac30f3-5974-4465-91ad-fe10eafa0c28 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:56.247655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac30f3-5974-4465-91ad-fe10eafa0c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:56.248120 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac30f3-5974-4465-91ad-fe10eafa0c28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:56.249055 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac30f3-5974-4465-91ad-fe10eafa0c28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:56.249442 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac30f3-5974-4465-91ad-fe10eafa0c28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:56.252091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac30f3-5974-4465-91ad-fe10eafa0c28 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:56.261383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1070/2123] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:56.265352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca700be-bc29-4e8f-8ae3-fb061bcce1db None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:56.307577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca700be-bc29-4e8f-8ae3-fb061bcce1db None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:56.314638 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1054/2124] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:56.317168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac30f3-5974-4465-91ad-fe10eafa0c28 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:56.318996 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23790a59-4af0-468b-85ea-75ca405dfe77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:56.320435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23790a59-4af0-468b-85ea-75ca405dfe77 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:56.321047 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23790a59-4af0-468b-85ea-75ca405dfe77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:56.321719 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23790a59-4af0-468b-85ea-75ca405dfe77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:56.323160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23790a59-4af0-468b-85ea-75ca405dfe77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:56.323792 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23790a59-4af0-468b-85ea-75ca405dfe77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:56.327325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23790a59-4af0-468b-85ea-75ca405dfe77 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:56.359600 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac30f3-5974-4465-91ad-fe10eafa0c28 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:56.383455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bc70f-4e81-4723-b265-5820435e30f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:56.385098 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bc70f-4e81-4723-b265-5820435e30f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:56.385819 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bc70f-4e81-4723-b265-5820435e30f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:56.386540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bc70f-4e81-4723-b265-5820435e30f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:56.388661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bc70f-4e81-4723-b265-5820435e30f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:56.393236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bc70f-4e81-4723-b265-5820435e30f8 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:56.413326 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1071/2125] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:56.418016 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23790a59-4af0-468b-85ea-75ca405dfe77 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:56.451891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23790a59-4af0-468b-85ea-75ca405dfe77 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:56.477323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61e1bb-3052-4790-bbe8-8a0723b20b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:56.480147 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61e1bb-3052-4790-bbe8-8a0723b20b89 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:56.480689 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61e1bb-3052-4790-bbe8-8a0723b20b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:56.483782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61e1bb-3052-4790-bbe8-8a0723b20b89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:56.485501 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61e1bb-3052-4790-bbe8-8a0723b20b89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:56.493326 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61e1bb-3052-4790-bbe8-8a0723b20b89 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:56.494212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1055/2126] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:56 2022] POST /identity/v3/projects => generated 391 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:56.500372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bc70f-4e81-4723-b265-5820435e30f8 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:56.638133 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1072/2127] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:56 2022] POST /identity/v3/projects => generated 381 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:56.640594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61e1bb-3052-4790-bbe8-8a0723b20b89 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:56.718968 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:56.778436 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:56.843397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bc70f-4e81-4723-b265-5820435e30f8 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:56.954885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61e1bb-3052-4790-bbe8-8a0723b20b89 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:56.972848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b07ab9-204a-489c-958c-13c9d74cbca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:56.973731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b07ab9-204a-489c-958c-13c9d74cbca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:56.976630 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b07ab9-204a-489c-958c-13c9d74cbca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:56.976985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b07ab9-204a-489c-958c-13c9d74cbca6 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:56.977832 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b07ab9-204a-489c-958c-13c9d74cbca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:56.989424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b07ab9-204a-489c-958c-13c9d74cbca6 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:57.020899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15af16a-241a-4cbf-a099-228ba4e5e3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:57.020899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15af16a-241a-4cbf-a099-228ba4e5e3f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:57.020899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15af16a-241a-4cbf-a099-228ba4e5e3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:57.020899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15af16a-241a-4cbf-a099-228ba4e5e3f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:57.021557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15af16a-241a-4cbf-a099-228ba4e5e3f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:57.021557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15af16a-241a-4cbf-a099-228ba4e5e3f8 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:57.040838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1073/2128] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:56 2022] POST /identity/v3/users => generated 291 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:57.041976 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b07ab9-204a-489c-958c-13c9d74cbca6 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:57.047184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-69b07ab9-204a-489c-958c-13c9d74cbca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:57.071990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b07ab9-204a-489c-958c-13c9d74cbca6 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:57.078265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07273582-19af-4fb7-9fed-7ae6ff6c7304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:57.078934 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07273582-19af-4fb7-9fed-7ae6ff6c7304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:57.079209 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07273582-19af-4fb7-9fed-7ae6ff6c7304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:57.079483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07273582-19af-4fb7-9fed-7ae6ff6c7304 admin admin] PATH_INFO: `/v3/projects/1567188877984156bb45e9f014ad2c24` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:57.081064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07273582-19af-4fb7-9fed-7ae6ff6c7304 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1567188877984156bb45e9f014ad2c24)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:57.083031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07273582-19af-4fb7-9fed-7ae6ff6c7304 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:57.091660 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1056/2129] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:56 2022] POST /identity/v3/users => generated 296 bytes in 596 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:57.093882 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15af16a-241a-4cbf-a099-228ba4e5e3f8 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:57.118474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15af16a-241a-4cbf-a099-228ba4e5e3f8 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:57.129368 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2f595-93a1-48a3-92e3-234765565ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:57.131801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2f595-93a1-48a3-92e3-234765565ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:57.133194 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2f595-93a1-48a3-92e3-234765565ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:57.133924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2f595-93a1-48a3-92e3-234765565ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:57.136324 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2f595-93a1-48a3-92e3-234765565ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:57.141153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2f595-93a1-48a3-92e3-234765565ed0 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:57.171766 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1057/2130] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:57.172541 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2f595-93a1-48a3-92e3-234765565ed0 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:57.196652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2f595-93a1-48a3-92e3-234765565ed0 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:57.212986 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce344d5f-ddf5-4e5a-949d-082da9f75270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:57.213743 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce344d5f-ddf5-4e5a-949d-082da9f75270 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:57.214037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce344d5f-ddf5-4e5a-949d-082da9f75270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:57.214315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce344d5f-ddf5-4e5a-949d-082da9f75270 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:57.217774 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce344d5f-ddf5-4e5a-949d-082da9f75270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:57.221541 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce344d5f-ddf5-4e5a-949d-082da9f75270 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:57.247911 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-07273582-19af-4fb7-9fed-7ae6ff6c7304 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1567188877984156bb45e9f014ad2c24', 'request_id': 'req-07273582-19af-4fb7-9fed-7ae6ff6c7304'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:57.254395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1058/2131] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:57.255361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce344d5f-ddf5-4e5a-949d-082da9f75270 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:57.267419 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-07273582-19af-4fb7-9fed-7ae6ff6c7304 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1567188877984156bb45e9f014ad2c24', 'request_id': 'req-07273582-19af-4fb7-9fed-7ae6ff6c7304'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:57.276026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce344d5f-ddf5-4e5a-949d-082da9f75270 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:57.298770 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-07273582-19af-4fb7-9fed-7ae6ff6c7304 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1567188877984156bb45e9f014ad2c24', 'request_id': 'req-07273582-19af-4fb7-9fed-7ae6ff6c7304'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:57.342325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-07273582-19af-4fb7-9fed-7ae6ff6c7304 admin admin] The token cache is being invalidate because project 1567188877984156bb45e9f014ad2c24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:57.343388 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-07273582-19af-4fb7-9fed-7ae6ff6c7304 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:57.344963 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-07273582-19af-4fb7-9fed-7ae6ff6c7304 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:57.351666 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1074/2132] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:14:57 2022] DELETE /identity/v3/projects/1567188877984156bb45e9f014ad2c24 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:57.356008 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07273582-19af-4fb7-9fed-7ae6ff6c7304 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:57.380722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-07273582-19af-4fb7-9fed-7ae6ff6c7304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:57.465670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ba9d1-f860-4b83-8283-c530c64e92ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:57.474003 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ba9d1-f860-4b83-8283-c530c64e92ed admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:57.475084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ba9d1-f860-4b83-8283-c530c64e92ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:57.476731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ba9d1-f860-4b83-8283-c530c64e92ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:57.484948 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ba9d1-f860-4b83-8283-c530c64e92ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:57.488431 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ba9d1-f860-4b83-8283-c530c64e92ed admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:57.572187 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1059/2133] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:57.574236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ba9d1-f860-4b83-8283-c530c64e92ed admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:57.580822 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e53ba9d1-f860-4b83-8283-c530c64e92ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:57.612670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ba9d1-f860-4b83-8283-c530c64e92ed admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:57.623721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d19f4-79cb-4e41-bacd-4b3ad504abcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:57.625996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d19f4-79cb-4e41-bacd-4b3ad504abcf admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:57.628235 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d19f4-79cb-4e41-bacd-4b3ad504abcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:57.629908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d19f4-79cb-4e41-bacd-4b3ad504abcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:57.633466 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d19f4-79cb-4e41-bacd-4b3ad504abcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:57.637480 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d19f4-79cb-4e41-bacd-4b3ad504abcf admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:57.670332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1060/2134] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:57.672342 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d19f4-79cb-4e41-bacd-4b3ad504abcf admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:57.673669 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07273582-19af-4fb7-9fed-7ae6ff6c7304 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:57.675496 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-8b3d19f4-79cb-4e41-bacd-4b3ad504abcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:57.699489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b395158-b699-4ec3-beef-30611374c896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:57.701516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b395158-b699-4ec3-beef-30611374c896 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:57.702046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b395158-b699-4ec3-beef-30611374c896 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:57.702531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b395158-b699-4ec3-beef-30611374c896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:57.706226 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-6b395158-b699-4ec3-beef-30611374c896 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:57.708386 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d19f4-79cb-4e41-bacd-4b3ad504abcf admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:57.720128 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c022c-9f85-4e70-9492-45430dc7152a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:57.721815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c022c-9f85-4e70-9492-45430dc7152a admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:57.722760 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c022c-9f85-4e70-9492-45430dc7152a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:57.723663 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c022c-9f85-4e70-9492-45430dc7152a admin admin] PATH_INFO: `/v3/projects/e767ea8cd3824e50879a3e65d829e458/users/967e5145915948ad94d6fac50a33e805/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:57.762882 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c022c-9f85-4e70-9492-45430dc7152a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e767ea8cd3824e50879a3e65d829e458, user_id=967e5145915948ad94d6fac50a33e805, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:57.766225 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c022c-9f85-4e70-9492-45430dc7152a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:57.799377 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1061/2135] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:57 2022] PUT /identity/v3/projects/e767ea8cd3824e50879a3e65d829e458/users/967e5145915948ad94d6fac50a33e805/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:57.801004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c022c-9f85-4e70-9492-45430dc7152a admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:57.826495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:57.844052 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:57.869369 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b395158-b699-4ec3-beef-30611374c896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:57.872452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b395158-b699-4ec3-beef-30611374c896 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:57.890724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1075/2136] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:57.892030 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7abdc4-21ed-4db1-9a4d-328524fab26c None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:57.892373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7abdc4-21ed-4db1-9a4d-328524fab26c None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:57.892684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7abdc4-21ed-4db1-9a4d-328524fab26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:57.954788 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-fa7abdc4-21ed-4db1-9a4d-328524fab26c None None] MFA Rules not processed for user `967e5145915948ad94d6fac50a33e805`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:57.958482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-fa7abdc4-21ed-4db1-9a4d-328524fab26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:58.009547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c022c-9f85-4e70-9492-45430dc7152a admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:58.017600 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb3149-2ce2-4bda-839a-b4c4a2e92fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:58.018535 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb3149-2ce2-4bda-839a-b4c4a2e92fff admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:58.019155 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb3149-2ce2-4bda-839a-b4c4a2e92fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:58.019481 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb3149-2ce2-4bda-839a-b4c4a2e92fff admin admin] PATH_INFO: `/v3/projects/bac59c62cb6c4943b193c9f586c29c8f/users/e47d2e62189b477f93e046b844195461/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:58.084450 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb3149-2ce2-4bda-839a-b4c4a2e92fff admin admin] RBAC: Authorizing `identity:create_grant(project_id=bac59c62cb6c4943b193c9f586c29c8f, user_id=e47d2e62189b477f93e046b844195461, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:58.091650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb3149-2ce2-4bda-839a-b4c4a2e92fff admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:58.146450 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1062/2137] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:57 2022] PUT /identity/v3/projects/bac59c62cb6c4943b193c9f586c29c8f/users/e47d2e62189b477f93e046b844195461/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:58.173718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb768e2-a60a-463e-a309-634956a5cc07 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:58.175147 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb768e2-a60a-463e-a309-634956a5cc07 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:58.176404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb768e2-a60a-463e-a309-634956a5cc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:58.188521 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1076/2138] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:58.192768 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7abdc4-21ed-4db1-9a4d-328524fab26c None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:58.228317 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7abdc4-21ed-4db1-9a4d-328524fab26c None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:58.269014 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-6cb768e2-a60a-463e-a309-634956a5cc07 None None] MFA Rules not processed for user `e47d2e62189b477f93e046b844195461`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:58.273412 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-6cb768e2-a60a-463e-a309-634956a5cc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:58.341851 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d3d14-9b5f-4d69-9cb7-dff82b31ecbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:58.343095 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d3d14-9b5f-4d69-9cb7-dff82b31ecbf admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:58.343095 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d3d14-9b5f-4d69-9cb7-dff82b31ecbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:58.343477 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d3d14-9b5f-4d69-9cb7-dff82b31ecbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:58.344899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d3d14-9b5f-4d69-9cb7-dff82b31ecbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:58.346993 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d3d14-9b5f-4d69-9cb7-dff82b31ecbf admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:58.402885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1077/2139] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:14:58 2022] POST /identity/v3/projects => generated 379 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:58.408782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d3d14-9b5f-4d69-9cb7-dff82b31ecbf admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:58.446361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1063/2140] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:58.462734 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb768e2-a60a-463e-a309-634956a5cc07 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:58.526214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:58.532928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d3d14-9b5f-4d69-9cb7-dff82b31ecbf admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:58.545235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656878a9-b053-4f4f-a75f-ad8276b75ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:58.546303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656878a9-b053-4f4f-a75f-ad8276b75ece admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:58.546781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656878a9-b053-4f4f-a75f-ad8276b75ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:58.547279 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656878a9-b053-4f4f-a75f-ad8276b75ece admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:58.548533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656878a9-b053-4f4f-a75f-ad8276b75ece admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:58.552428 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656878a9-b053-4f4f-a75f-ad8276b75ece admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:58.581683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb768e2-a60a-463e-a309-634956a5cc07 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:58.615309 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1078/2141] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:58 2022] POST /identity/v3/users => generated 296 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:58.621436 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656878a9-b053-4f4f-a75f-ad8276b75ece admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:58.651805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:58.657934 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656878a9-b053-4f4f-a75f-ad8276b75ece admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:58.684683 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523214d1-e857-4998-abb2-0605689de968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:58.685720 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523214d1-e857-4998-abb2-0605689de968 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:58.686213 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523214d1-e857-4998-abb2-0605689de968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:58.686684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523214d1-e857-4998-abb2-0605689de968 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:58.687904 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523214d1-e857-4998-abb2-0605689de968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:58.688963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d91415-0923-4354-8834-9ff27e841f57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:58.690363 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d91415-0923-4354-8834-9ff27e841f57 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:58.690967 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d91415-0923-4354-8834-9ff27e841f57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:58.691583 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d91415-0923-4354-8834-9ff27e841f57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:58.696031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523214d1-e857-4998-abb2-0605689de968 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:58.744008 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1079/2142] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:58.746860 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523214d1-e857-4998-abb2-0605689de968 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:58.794453 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523214d1-e857-4998-abb2-0605689de968 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:58.810131 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8d0d1-1758-4aff-802c-124ef068b169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:58.811289 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8d0d1-1758-4aff-802c-124ef068b169 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:58.811777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8d0d1-1758-4aff-802c-124ef068b169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:58.812279 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8d0d1-1758-4aff-802c-124ef068b169 admin admin] PATH_INFO: `/v3/projects/2e93ca84c7e4403f8d6e0cb1c1314b88/users/ec4d386e729c4aafaa537c9dff61e6f3/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:58.850034 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8d0d1-1758-4aff-802c-124ef068b169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e93ca84c7e4403f8d6e0cb1c1314b88, user_id=ec4d386e729c4aafaa537c9dff61e6f3, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:58.862555 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8d0d1-1758-4aff-802c-124ef068b169 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:58.897237 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1080/2143] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:58 2022] PUT /identity/v3/projects/2e93ca84c7e4403f8d6e0cb1c1314b88/users/ec4d386e729c4aafaa537c9dff61e6f3/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:58.902286 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d91415-0923-4354-8834-9ff27e841f57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:58.905706 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d91415-0923-4354-8834-9ff27e841f57 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:58.924432 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1064/2144] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:58.937256 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8d0d1-1758-4aff-802c-124ef068b169 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:59.185872 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8d0d1-1758-4aff-802c-124ef068b169 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:59.198353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee1207-9f7d-473f-aeda-b41277c57a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:59.199484 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee1207-9f7d-473f-aeda-b41277c57a98 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:59.199988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee1207-9f7d-473f-aeda-b41277c57a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:59.200491 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee1207-9f7d-473f-aeda-b41277c57a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:59.201894 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee1207-9f7d-473f-aeda-b41277c57a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:59.205211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee1207-9f7d-473f-aeda-b41277c57a98 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:59.240057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d91415-0923-4354-8834-9ff27e841f57 service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:59.249525 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1081/2145] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:14:59.262469 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d91415-0923-4354-8834-9ff27e841f57 service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:59.272933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e5ecd-0393-4a2d-bb37-e76d39f679dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:59.274196 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e5ecd-0393-4a2d-bb37-e76d39f679dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:59.274774 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e5ecd-0393-4a2d-bb37-e76d39f679dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:59.275664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e5ecd-0393-4a2d-bb37-e76d39f679dc admin admin] PATH_INFO: `/v3/projects/2e93ca84c7e4403f8d6e0cb1c1314b88/users/ec4d386e729c4aafaa537c9dff61e6f3/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:59.284281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e5ecd-0393-4a2d-bb37-e76d39f679dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e93ca84c7e4403f8d6e0cb1c1314b88, user_id=ec4d386e729c4aafaa537c9dff61e6f3, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:59.302095 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e5ecd-0393-4a2d-bb37-e76d39f679dc admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:59.346446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1065/2146] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:14:59 2022] PUT /identity/v3/projects/2e93ca84c7e4403f8d6e0cb1c1314b88/users/ec4d386e729c4aafaa537c9dff61e6f3/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:59.370699 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad914f60-2cfb-47e8-b2ee-8f40d998db72 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:59.371330 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad914f60-2cfb-47e8-b2ee-8f40d998db72 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:59.372536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad914f60-2cfb-47e8-b2ee-8f40d998db72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:59.473710 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-ad914f60-2cfb-47e8-b2ee-8f40d998db72 None None] MFA Rules not processed for user `ec4d386e729c4aafaa537c9dff61e6f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:59.477918 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ad914f60-2cfb-47e8-b2ee-8f40d998db72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:59.833819 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1082/2147] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:14:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:59.922894 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e5ecd-0393-4a2d-bb37-e76d39f679dc admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:00.075558 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e5ecd-0393-4a2d-bb37-e76d39f679dc admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:00.089672 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef9cd3-1744-4d34-a864-6308df615193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:00.092912 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef9cd3-1744-4d34-a864-6308df615193 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:00.095575 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef9cd3-1744-4d34-a864-6308df615193 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:00.101626 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef9cd3-1744-4d34-a864-6308df615193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:00.135477 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ef9cd3-1744-4d34-a864-6308df615193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:00.139155 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ef9cd3-1744-4d34-a864-6308df615193 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:00.167735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1066/2148] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:14:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:00.370148 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad914f60-2cfb-47e8-b2ee-8f40d998db72 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:00.402480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad914f60-2cfb-47e8-b2ee-8f40d998db72 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:00.410204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433dff8-5710-42eb-8a00-a024c8b9e2e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:00.418139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433dff8-5710-42eb-8a00-a024c8b9e2e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:00.418530 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433dff8-5710-42eb-8a00-a024c8b9e2e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:00.418906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433dff8-5710-42eb-8a00-a024c8b9e2e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:00.527990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:00.584306 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433dff8-5710-42eb-8a00-a024c8b9e2e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:00.587932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433dff8-5710-42eb-8a00-a024c8b9e2e4 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:00.608135 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1083/2149] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:00.916658 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11979066-e00f-4d64-b8ef-84ad15e1ebdb None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:00.917677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11979066-e00f-4d64-b8ef-84ad15e1ebdb None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:00.918585 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11979066-e00f-4d64-b8ef-84ad15e1ebdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:00.965428 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-11979066-e00f-4d64-b8ef-84ad15e1ebdb None None] MFA Rules not processed for user `16f05877e10442d2a8938f6805eff842`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:00.967485 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-11979066-e00f-4d64-b8ef-84ad15e1ebdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:01.084634 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1067/2150] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:05.443663 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433dff8-5710-42eb-8a00-a024c8b9e2e4 service cinder] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:05.445811 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e433dff8-5710-42eb-8a00-a024c8b9e2e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:05.571068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433dff8-5710-42eb-8a00-a024c8b9e2e4 service cinder] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:05.580666 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443abbb-7cce-40ef-8b7f-772a5daa87c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:05.581929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443abbb-7cce-40ef-8b7f-772a5daa87c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:05.587004 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443abbb-7cce-40ef-8b7f-772a5daa87c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:05.587686 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443abbb-7cce-40ef-8b7f-772a5daa87c3 admin admin] PATH_INFO: `/v3/users/56a746907610432195d4bfdc6bc09937` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:05.590328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443abbb-7cce-40ef-8b7f-772a5daa87c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56a746907610432195d4bfdc6bc09937)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:05.599079 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443abbb-7cce-40ef-8b7f-772a5daa87c3 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:05.709343 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d443abbb-7cce-40ef-8b7f-772a5daa87c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56a746907610432195d4bfdc6bc09937'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:05.725288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d443abbb-7cce-40ef-8b7f-772a5daa87c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56a746907610432195d4bfdc6bc09937'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:05.739464 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d443abbb-7cce-40ef-8b7f-772a5daa87c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56a746907610432195d4bfdc6bc09937'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:05.740522 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d443abbb-7cce-40ef-8b7f-772a5daa87c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56a746907610432195d4bfdc6bc09937'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:05.741612 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d443abbb-7cce-40ef-8b7f-772a5daa87c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56a746907610432195d4bfdc6bc09937'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:05.775301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1084/2151] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:15:05 2022] DELETE /identity/v3/users/56a746907610432195d4bfdc6bc09937 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:05.781419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90d0ec-9d3b-4d71-911e-e3b9d52ccda0 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:05.782366 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90d0ec-9d3b-4d71-911e-e3b9d52ccda0 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:05.783075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90d0ec-9d3b-4d71-911e-e3b9d52ccda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:05.832240 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-6e90d0ec-9d3b-4d71-911e-e3b9d52ccda0 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:05.834628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-6e90d0ec-9d3b-4d71-911e-e3b9d52ccda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:05.875926 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:05.987964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1068/2152] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:06.003808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443abbb-7cce-40ef-8b7f-772a5daa87c3 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:06.006597 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-d443abbb-7cce-40ef-8b7f-772a5daa87c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:06.100731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443abbb-7cce-40ef-8b7f-772a5daa87c3 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:06.107945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b0400-02f6-48be-a3ba-22eaaf95b33f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:06.109027 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b0400-02f6-48be-a3ba-22eaaf95b33f service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:06.109551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b0400-02f6-48be-a3ba-22eaaf95b33f service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:06.110025 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b0400-02f6-48be-a3ba-22eaaf95b33f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:06.121370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:06.130258 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b0400-02f6-48be-a3ba-22eaaf95b33f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:06.133306 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b0400-02f6-48be-a3ba-22eaaf95b33f service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:06.152102 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1085/2153] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:15:06.892234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90d0ec-9d3b-4d71-911e-e3b9d52ccda0 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:06.894679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-6e90d0ec-9d3b-4d71-911e-e3b9d52ccda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:06.931968 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90d0ec-9d3b-4d71-911e-e3b9d52ccda0 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:06.947061 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a8e12-fc99-4212-84a9-327af407f2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:06.960912 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a8e12-fc99-4212-84a9-327af407f2de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:06.961807 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a8e12-fc99-4212-84a9-327af407f2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:06.962471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a8e12-fc99-4212-84a9-327af407f2de admin admin] PATH_INFO: `/v3/projects/c09a0a659b0640c0884544172763e034` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:06.965537 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a8e12-fc99-4212-84a9-327af407f2de admin admin] RBAC: Authorizing `identity:delete_project(project_id=c09a0a659b0640c0884544172763e034)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:06.969286 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a8e12-fc99-4212-84a9-327af407f2de admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:07.071421 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-818a8e12-fc99-4212-84a9-327af407f2de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c09a0a659b0640c0884544172763e034', 'request_id': 'req-818a8e12-fc99-4212-84a9-327af407f2de'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:07.084392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-818a8e12-fc99-4212-84a9-327af407f2de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c09a0a659b0640c0884544172763e034', 'request_id': 'req-818a8e12-fc99-4212-84a9-327af407f2de'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:07.094333 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-818a8e12-fc99-4212-84a9-327af407f2de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c09a0a659b0640c0884544172763e034', 'request_id': 'req-818a8e12-fc99-4212-84a9-327af407f2de'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:07.118844 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-818a8e12-fc99-4212-84a9-327af407f2de admin admin] The token cache is being invalidate because project c09a0a659b0640c0884544172763e034 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:07.119643 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-818a8e12-fc99-4212-84a9-327af407f2de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:07.120720 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-818a8e12-fc99-4212-84a9-327af407f2de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:07.123942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1069/2154] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:15:06 2022] DELETE /identity/v3/projects/c09a0a659b0640c0884544172763e034 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:07.129097 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b0400-02f6-48be-a3ba-22eaaf95b33f service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:07.145730 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b0400-02f6-48be-a3ba-22eaaf95b33f service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:07.261686 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98480461-339d-4b4c-b14f-29160fd508bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:07.262797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98480461-339d-4b4c-b14f-29160fd508bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:07.263297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98480461-339d-4b4c-b14f-29160fd508bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:07.263785 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98480461-339d-4b4c-b14f-29160fd508bd admin admin] PATH_INFO: `/v3/users/b7e29fa765434350980531b0d1aedbaa` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:07.266197 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98480461-339d-4b4c-b14f-29160fd508bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7e29fa765434350980531b0d1aedbaa)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:07.270121 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98480461-339d-4b4c-b14f-29160fd508bd admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:07.360075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-98480461-339d-4b4c-b14f-29160fd508bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7e29fa765434350980531b0d1aedbaa'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:07.375436 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-98480461-339d-4b4c-b14f-29160fd508bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7e29fa765434350980531b0d1aedbaa'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:07.389081 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-98480461-339d-4b4c-b14f-29160fd508bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7e29fa765434350980531b0d1aedbaa'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:07.389931 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-98480461-339d-4b4c-b14f-29160fd508bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7e29fa765434350980531b0d1aedbaa'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:07.390543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-98480461-339d-4b4c-b14f-29160fd508bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7e29fa765434350980531b0d1aedbaa'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:07.418726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1086/2155] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:15:07 2022] DELETE /identity/v3/users/b7e29fa765434350980531b0d1aedbaa => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:07.858627 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a8e12-fc99-4212-84a9-327af407f2de admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:07.861343 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-818a8e12-fc99-4212-84a9-327af407f2de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:07.894376 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a8e12-fc99-4212-84a9-327af407f2de admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:08.026807 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd58d12-8d9b-4727-ae8d-1fd953a6cce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:08.028122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd58d12-8d9b-4727-ae8d-1fd953a6cce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:08.030742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd58d12-8d9b-4727-ae8d-1fd953a6cce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:08.030742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd58d12-8d9b-4727-ae8d-1fd953a6cce3 admin admin] PATH_INFO: `/v3/projects/1d37787a60e74327ac4a19b4a6413f94` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:08.032123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd58d12-8d9b-4727-ae8d-1fd953a6cce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d37787a60e74327ac4a19b4a6413f94)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:08.036854 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd58d12-8d9b-4727-ae8d-1fd953a6cce3 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:08.121995 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1cd58d12-8d9b-4727-ae8d-1fd953a6cce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d37787a60e74327ac4a19b4a6413f94', 'request_id': 'req-1cd58d12-8d9b-4727-ae8d-1fd953a6cce3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.126107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:08.138171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1cd58d12-8d9b-4727-ae8d-1fd953a6cce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d37787a60e74327ac4a19b4a6413f94', 'request_id': 'req-1cd58d12-8d9b-4727-ae8d-1fd953a6cce3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.151150 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1cd58d12-8d9b-4727-ae8d-1fd953a6cce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d37787a60e74327ac4a19b4a6413f94', 'request_id': 'req-1cd58d12-8d9b-4727-ae8d-1fd953a6cce3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.171214 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1cd58d12-8d9b-4727-ae8d-1fd953a6cce3 admin admin] The token cache is being invalidate because project 1d37787a60e74327ac4a19b4a6413f94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:08.171682 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1cd58d12-8d9b-4727-ae8d-1fd953a6cce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.172887 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1cd58d12-8d9b-4727-ae8d-1fd953a6cce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.175644 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1070/2156] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:15:07 2022] DELETE /identity/v3/projects/1d37787a60e74327ac4a19b4a6413f94 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:08.182052 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98480461-339d-4b4c-b14f-29160fd508bd admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:08.187128 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-98480461-339d-4b4c-b14f-29160fd508bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:08.216610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98480461-339d-4b4c-b14f-29160fd508bd admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:08.338386 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d49e3-cda5-4fde-8c50-b9530015aa8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:08.340763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d49e3-cda5-4fde-8c50-b9530015aa8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:08.341316 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d49e3-cda5-4fde-8c50-b9530015aa8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:08.341642 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d49e3-cda5-4fde-8c50-b9530015aa8c admin admin] PATH_INFO: `/v3/users/bbc20ae691c3461d9a8741c0f4881d51` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:08.343220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d49e3-cda5-4fde-8c50-b9530015aa8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbc20ae691c3461d9a8741c0f4881d51)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:08.345420 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d49e3-cda5-4fde-8c50-b9530015aa8c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:08.369232 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:08.436864 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dff5cf-ca86-4f16-a999-e3adcf2472f3 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:08.437365 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dff5cf-ca86-4f16-a999-e3adcf2472f3 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:08.437988 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dff5cf-ca86-4f16-a999-e3adcf2472f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:08.455855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-745d49e3-cda5-4fde-8c50-b9530015aa8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbc20ae691c3461d9a8741c0f4881d51'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.475742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-e0dff5cf-ca86-4f16-a999-e3adcf2472f3 None None] MFA Rules not processed for user `4370f8f474c2446e989d0d8c5b75e87a`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:08.477321 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e0dff5cf-ca86-4f16-a999-e3adcf2472f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:08.478329 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-745d49e3-cda5-4fde-8c50-b9530015aa8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbc20ae691c3461d9a8741c0f4881d51'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.502675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-745d49e3-cda5-4fde-8c50-b9530015aa8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbc20ae691c3461d9a8741c0f4881d51'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.503976 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-745d49e3-cda5-4fde-8c50-b9530015aa8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbc20ae691c3461d9a8741c0f4881d51'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.505014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-745d49e3-cda5-4fde-8c50-b9530015aa8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbc20ae691c3461d9a8741c0f4881d51'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.568753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1087/2157] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:15:08 2022] DELETE /identity/v3/users/bbc20ae691c3461d9a8741c0f4881d51 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:08.625369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1071/2158] 10.222.0.93 () {58 vars in 1009 bytes} [Mon Aug 1 15:15:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:08.648369 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d49e3-cda5-4fde-8c50-b9530015aa8c admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:08.651204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-745d49e3-cda5-4fde-8c50-b9530015aa8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:08.802539 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d49e3-cda5-4fde-8c50-b9530015aa8c admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:08.812404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef5ee3-3b48-4f47-bc0c-badec0d4085e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4df37b02152c48509243b73ebbf49d8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:08.813655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef5ee3-3b48-4f47-bc0c-badec0d4085e service swift] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:08.814149 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef5ee3-3b48-4f47-bc0c-badec0d4085e service swift] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:08.814642 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef5ee3-3b48-4f47-bc0c-badec0d4085e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:08.943183 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef5ee3-3b48-4f47-bc0c-badec0d4085e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:08.947120 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef5ee3-3b48-4f47-bc0c-badec0d4085e service swift] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:08.965719 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1088/2159] 10.222.0.93 () {60 vars in 1384 bytes} [Mon Aug 1 15:15:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:15:09.138324 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dff5cf-ca86-4f16-a999-e3adcf2472f3 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:09.140146 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e0dff5cf-ca86-4f16-a999-e3adcf2472f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:09.227017 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dff5cf-ca86-4f16-a999-e3adcf2472f3 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:09.403662 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1896c86-2411-4622-bf75-e74f13e19662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:09.405275 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1896c86-2411-4622-bf75-e74f13e19662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:09.406010 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1896c86-2411-4622-bf75-e74f13e19662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:09.406832 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1896c86-2411-4622-bf75-e74f13e19662 admin admin] PATH_INFO: `/v3/projects/ad0b809524224af2ac8bd128589e9ff1` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:09.410068 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1896c86-2411-4622-bf75-e74f13e19662 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad0b809524224af2ac8bd128589e9ff1)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:09.416769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1896c86-2411-4622-bf75-e74f13e19662 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:09.607227 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e1896c86-2411-4622-bf75-e74f13e19662 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad0b809524224af2ac8bd128589e9ff1', 'request_id': 'req-e1896c86-2411-4622-bf75-e74f13e19662'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:09.647745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e1896c86-2411-4622-bf75-e74f13e19662 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad0b809524224af2ac8bd128589e9ff1', 'request_id': 'req-e1896c86-2411-4622-bf75-e74f13e19662'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:09.705498 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e1896c86-2411-4622-bf75-e74f13e19662 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad0b809524224af2ac8bd128589e9ff1', 'request_id': 'req-e1896c86-2411-4622-bf75-e74f13e19662'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:09.774089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e1896c86-2411-4622-bf75-e74f13e19662 admin admin] The token cache is being invalidate because project ad0b809524224af2ac8bd128589e9ff1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:09.777297 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e1896c86-2411-4622-bf75-e74f13e19662 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:09.778923 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-e1896c86-2411-4622-bf75-e74f13e19662 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:09.790896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1072/2160] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:15:09 2022] DELETE /identity/v3/projects/ad0b809524224af2ac8bd128589e9ff1 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:09.791171 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4d1b2-59c8-4646-8ca1-98c107abc584 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:09.791883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4d1b2-59c8-4646-8ca1-98c107abc584 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:09.791883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4d1b2-59c8-4646-8ca1-98c107abc584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:09.832318 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-b1a4d1b2-59c8-4646-8ca1-98c107abc584 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:09.834198 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b1a4d1b2-59c8-4646-8ca1-98c107abc584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:09.874578 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:10.016434 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1089/2161] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:10.031333 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7e005-839f-4b09-99da-821fb399549c None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:10.031685 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7e005-839f-4b09-99da-821fb399549c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:10.032158 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7e005-839f-4b09-99da-821fb399549c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:10.072146 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-a3e7e005-839f-4b09-99da-821fb399549c None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:10.074039 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a3e7e005-839f-4b09-99da-821fb399549c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:10.089691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4d1b2-59c8-4646-8ca1-98c107abc584 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:10.096448 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1073/2162] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:10.106111 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4d1b2-59c8-4646-8ca1-98c107abc584 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:10.113439 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf206d8e-89a4-4b54-9102-d730d4126c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:10.113787 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf206d8e-89a4-4b54-9102-d730d4126c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:10.113967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf206d8e-89a4-4b54-9102-d730d4126c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:10.114135 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf206d8e-89a4-4b54-9102-d730d4126c2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:10.115308 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf206d8e-89a4-4b54-9102-d730d4126c2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:15:10.115687 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf206d8e-89a4-4b54-9102-d730d4126c2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:10.118376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf206d8e-89a4-4b54-9102-d730d4126c2a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:10.149187 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1090/2163] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:15:10.154380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7e005-839f-4b09-99da-821fb399549c None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:10.176801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7e005-839f-4b09-99da-821fb399549c None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:10.203637 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7915099-2bce-4a5a-b9a7-84f85a46f15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:10.205280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7915099-2bce-4a5a-b9a7-84f85a46f15c admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:10.205736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7915099-2bce-4a5a-b9a7-84f85a46f15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:10.206225 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7915099-2bce-4a5a-b9a7-84f85a46f15c admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:10.210625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7915099-2bce-4a5a-b9a7-84f85a46f15c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:10.231265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7915099-2bce-4a5a-b9a7-84f85a46f15c admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:10.246281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:10.310574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf206d8e-89a4-4b54-9102-d730d4126c2a admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:10.318141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1074/2164] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:15:10 2022] POST /identity/v3/projects => generated 365 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:10.337315 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf206d8e-89a4-4b54-9102-d730d4126c2a admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:10.451731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a7aa1-7611-468b-a20c-b44db332f305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:10.452357 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a7aa1-7611-468b-a20c-b44db332f305 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:10.452766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a7aa1-7611-468b-a20c-b44db332f305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:10.453548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a7aa1-7611-468b-a20c-b44db332f305 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:10.454116 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a7aa1-7611-468b-a20c-b44db332f305 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:10.456520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a7aa1-7611-468b-a20c-b44db332f305 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:10.510833 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1091/2165] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:15:10 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:10.518592 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7915099-2bce-4a5a-b9a7-84f85a46f15c admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:10.537790 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7915099-2bce-4a5a-b9a7-84f85a46f15c admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:10.545972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fc45f-776e-4d0a-9646-e5ebf70fbbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:10.547049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fc45f-776e-4d0a-9646-e5ebf70fbbc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:10.547553 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fc45f-776e-4d0a-9646-e5ebf70fbbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:10.548049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fc45f-776e-4d0a-9646-e5ebf70fbbc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:10.549525 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fc45f-776e-4d0a-9646-e5ebf70fbbc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:10.552842 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fc45f-776e-4d0a-9646-e5ebf70fbbc4 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:10.576859 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1075/2166] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:10.583545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a7aa1-7611-468b-a20c-b44db332f305 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:10.606905 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a7aa1-7611-468b-a20c-b44db332f305 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:10.615047 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd63773-f4bc-4b27-9491-27bb28780ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:10.616922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd63773-f4bc-4b27-9491-27bb28780ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:10.616922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd63773-f4bc-4b27-9491-27bb28780ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:10.616922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd63773-f4bc-4b27-9491-27bb28780ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:10.617539 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd63773-f4bc-4b27-9491-27bb28780ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:10.619487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd63773-f4bc-4b27-9491-27bb28780ca1 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:10.638918 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1092/2167] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:10.643416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fc45f-776e-4d0a-9646-e5ebf70fbbc4 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:10.657688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fc45f-776e-4d0a-9646-e5ebf70fbbc4 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:10.666338 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2bf06-9b49-47a3-8fa6-bdb1317511c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:10.667628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2bf06-9b49-47a3-8fa6-bdb1317511c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:10.667972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2bf06-9b49-47a3-8fa6-bdb1317511c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:10.668545 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2bf06-9b49-47a3-8fa6-bdb1317511c7 admin admin] PATH_INFO: `/v3/projects/7960c549fa3d43bcb7b29e9046767b2a/users/242563abc4214eb4a0f34ca4a320166c/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:10.701256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f2bf06-9b49-47a3-8fa6-bdb1317511c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7960c549fa3d43bcb7b29e9046767b2a, user_id=242563abc4214eb4a0f34ca4a320166c, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:10.705207 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f2bf06-9b49-47a3-8fa6-bdb1317511c7 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:10.729650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1076/2168] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:15:10 2022] PUT /identity/v3/projects/7960c549fa3d43bcb7b29e9046767b2a/users/242563abc4214eb4a0f34ca4a320166c/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:10.744234 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777a240-43ce-42f9-8e04-d04e679e03a5 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:10.744553 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777a240-43ce-42f9-8e04-d04e679e03a5 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:10.744834 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777a240-43ce-42f9-8e04-d04e679e03a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:10.787144 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-7777a240-43ce-42f9-8e04-d04e679e03a5 None None] MFA Rules not processed for user `242563abc4214eb4a0f34ca4a320166c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:10.789242 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-7777a240-43ce-42f9-8e04-d04e679e03a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:10.932546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1093/2169] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:10.949247 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2bf06-9b49-47a3-8fa6-bdb1317511c7 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:10.953686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-32f2bf06-9b49-47a3-8fa6-bdb1317511c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:11.043991 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2bf06-9b49-47a3-8fa6-bdb1317511c7 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:11.048865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7533008-31a5-43da-b81f-512c47199d5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:11.049471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7533008-31a5-43da-b81f-512c47199d5f service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:11.049731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7533008-31a5-43da-b81f-512c47199d5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:11.050113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7533008-31a5-43da-b81f-512c47199d5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:11.068746 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7533008-31a5-43da-b81f-512c47199d5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:11.071469 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7533008-31a5-43da-b81f-512c47199d5f service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:11.082719 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1077/2170] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:10 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:18.137368 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777a240-43ce-42f9-8e04-d04e679e03a5 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:18.140065 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-7777a240-43ce-42f9-8e04-d04e679e03a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:18.231482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:18.416356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777a240-43ce-42f9-8e04-d04e679e03a5 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:18.429809 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addae80f-59d4-416b-978c-52d5604c1f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:18.431264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addae80f-59d4-416b-978c-52d5604c1f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:18.455098 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addae80f-59d4-416b-978c-52d5604c1f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:18.457331 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addae80f-59d4-416b-978c-52d5604c1f94 admin admin] PATH_INFO: `/v3/users/967e5145915948ad94d6fac50a33e805` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:18.468245 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addae80f-59d4-416b-978c-52d5604c1f94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=967e5145915948ad94d6fac50a33e805)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:18.472094 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addae80f-59d4-416b-978c-52d5604c1f94 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:18.680713 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-addae80f-59d4-416b-978c-52d5604c1f94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '967e5145915948ad94d6fac50a33e805'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:18.722852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-addae80f-59d4-416b-978c-52d5604c1f94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '967e5145915948ad94d6fac50a33e805'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:18.742895 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-addae80f-59d4-416b-978c-52d5604c1f94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '967e5145915948ad94d6fac50a33e805'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:18.744637 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-addae80f-59d4-416b-978c-52d5604c1f94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '967e5145915948ad94d6fac50a33e805'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:18.759477 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-addae80f-59d4-416b-978c-52d5604c1f94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '967e5145915948ad94d6fac50a33e805'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:18.848984 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1094/2171] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:15:18 2022] DELETE /identity/v3/users/967e5145915948ad94d6fac50a33e805 => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:18.850937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c0317-4400-443f-9ef8-61cefcd2afe3 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:18.852546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c0317-4400-443f-9ef8-61cefcd2afe3 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:18.852546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c0317-4400-443f-9ef8-61cefcd2afe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:18.901531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-352c0317-4400-443f-9ef8-61cefcd2afe3 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:18.903281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-352c0317-4400-443f-9ef8-61cefcd2afe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:19.092664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1078/2172] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:19.123375 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addae80f-59d4-416b-978c-52d5604c1f94 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:19.132813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-addae80f-59d4-416b-978c-52d5604c1f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:19.313518 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addae80f-59d4-416b-978c-52d5604c1f94 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:19.326402 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12972bef-789f-4eb9-bab5-3b17955b63b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:19.330903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12972bef-789f-4eb9-bab5-3b17955b63b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:19.331885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12972bef-789f-4eb9-bab5-3b17955b63b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:19.332171 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12972bef-789f-4eb9-bab5-3b17955b63b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:19.363937 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12972bef-789f-4eb9-bab5-3b17955b63b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:19.369870 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12972bef-789f-4eb9-bab5-3b17955b63b4 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:19.434372 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1095/2173] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:20.693060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c0317-4400-443f-9ef8-61cefcd2afe3 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:20.699135 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-352c0317-4400-443f-9ef8-61cefcd2afe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:20.743727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c0317-4400-443f-9ef8-61cefcd2afe3 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:20.762507 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf8d00-879b-4a2b-a843-8e5a6a388679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:20.764949 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf8d00-879b-4a2b-a843-8e5a6a388679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:20.765982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf8d00-879b-4a2b-a843-8e5a6a388679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:20.767108 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf8d00-879b-4a2b-a843-8e5a6a388679 admin admin] PATH_INFO: `/v3/projects/e767ea8cd3824e50879a3e65d829e458` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:20.771000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbf8d00-879b-4a2b-a843-8e5a6a388679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e767ea8cd3824e50879a3e65d829e458)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:20.776199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbf8d00-879b-4a2b-a843-8e5a6a388679 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:20.904523 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-edbf8d00-879b-4a2b-a843-8e5a6a388679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e767ea8cd3824e50879a3e65d829e458', 'request_id': 'req-edbf8d00-879b-4a2b-a843-8e5a6a388679'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:20.907537 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:20.925970 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-edbf8d00-879b-4a2b-a843-8e5a6a388679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e767ea8cd3824e50879a3e65d829e458', 'request_id': 'req-edbf8d00-879b-4a2b-a843-8e5a6a388679'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:20.946542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-edbf8d00-879b-4a2b-a843-8e5a6a388679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e767ea8cd3824e50879a3e65d829e458', 'request_id': 'req-edbf8d00-879b-4a2b-a843-8e5a6a388679'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:20.981570 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-edbf8d00-879b-4a2b-a843-8e5a6a388679 admin admin] The token cache is being invalidate because project e767ea8cd3824e50879a3e65d829e458 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:20.982950 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-edbf8d00-879b-4a2b-a843-8e5a6a388679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:20.984769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-edbf8d00-879b-4a2b-a843-8e5a6a388679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:20.988434 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1079/2174] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:15:20 2022] DELETE /identity/v3/projects/e767ea8cd3824e50879a3e65d829e458 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:20.995604 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12972bef-789f-4eb9-bab5-3b17955b63b4 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:21.017025 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12972bef-789f-4eb9-bab5-3b17955b63b4 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:21.155442 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:21.187027 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d88a2-0697-40e9-8964-25b262223a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:21.188728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d88a2-0697-40e9-8964-25b262223a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:21.189396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d88a2-0697-40e9-8964-25b262223a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:21.190047 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d88a2-0697-40e9-8964-25b262223a0c admin admin] PATH_INFO: `/v3/users/ec4d386e729c4aafaa537c9dff61e6f3` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:21.193398 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d88a2-0697-40e9-8964-25b262223a0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec4d386e729c4aafaa537c9dff61e6f3)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:21.204841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d88a2-0697-40e9-8964-25b262223a0c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:21.353306 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-0c5d88a2-0697-40e9-8964-25b262223a0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec4d386e729c4aafaa537c9dff61e6f3'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:21.366039 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-0c5d88a2-0697-40e9-8964-25b262223a0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec4d386e729c4aafaa537c9dff61e6f3'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:21.388397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-0c5d88a2-0697-40e9-8964-25b262223a0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec4d386e729c4aafaa537c9dff61e6f3'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:21.390745 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-0c5d88a2-0697-40e9-8964-25b262223a0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec4d386e729c4aafaa537c9dff61e6f3'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:21.391949 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-0c5d88a2-0697-40e9-8964-25b262223a0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec4d386e729c4aafaa537c9dff61e6f3'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:21.431792 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1096/2175] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:15:20 2022] DELETE /identity/v3/users/ec4d386e729c4aafaa537c9dff61e6f3 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:22.187011 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf8d00-879b-4a2b-a843-8e5a6a388679 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:22.190150 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-edbf8d00-879b-4a2b-a843-8e5a6a388679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:22.581219 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf8d00-879b-4a2b-a843-8e5a6a388679 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:22.778065 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163647d-0148-4321-b7b7-ad7cfa5b277c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07e3e25e6a8466fb237dcb836aecda2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:22.779361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163647d-0148-4321-b7b7-ad7cfa5b277c service placement] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:22.779722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163647d-0148-4321-b7b7-ad7cfa5b277c service placement] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:22.780230 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163647d-0148-4321-b7b7-ad7cfa5b277c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:22.787383 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b163647d-0148-4321-b7b7-ad7cfa5b277c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:22.869726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163647d-0148-4321-b7b7-ad7cfa5b277c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:22.874925 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163647d-0148-4321-b7b7-ad7cfa5b277c service placement] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:22.936706 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1080/2176] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 750 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:23.013110 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d88a2-0697-40e9-8964-25b262223a0c admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:23.015511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0c5d88a2-0697-40e9-8964-25b262223a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:23.261900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d88a2-0697-40e9-8964-25b262223a0c admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:23.301014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc41b8d-3515-47cb-8920-761c8c9aa1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:23.302894 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc41b8d-3515-47cb-8920-761c8c9aa1b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:23.310620 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc41b8d-3515-47cb-8920-761c8c9aa1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:23.311343 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc41b8d-3515-47cb-8920-761c8c9aa1b3 admin admin] PATH_INFO: `/v3/projects/2e93ca84c7e4403f8d6e0cb1c1314b88` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:23.319239 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc41b8d-3515-47cb-8920-761c8c9aa1b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e93ca84c7e4403f8d6e0cb1c1314b88)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:23.328417 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc41b8d-3515-47cb-8920-761c8c9aa1b3 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:23.470984 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-bdc41b8d-3515-47cb-8920-761c8c9aa1b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e93ca84c7e4403f8d6e0cb1c1314b88', 'request_id': 'req-bdc41b8d-3515-47cb-8920-761c8c9aa1b3'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.492687 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-bdc41b8d-3515-47cb-8920-761c8c9aa1b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e93ca84c7e4403f8d6e0cb1c1314b88', 'request_id': 'req-bdc41b8d-3515-47cb-8920-761c8c9aa1b3'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.511809 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-bdc41b8d-3515-47cb-8920-761c8c9aa1b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e93ca84c7e4403f8d6e0cb1c1314b88', 'request_id': 'req-bdc41b8d-3515-47cb-8920-761c8c9aa1b3'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.531716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:23.541273 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-bdc41b8d-3515-47cb-8920-761c8c9aa1b3 admin admin] The token cache is being invalidate because project 2e93ca84c7e4403f8d6e0cb1c1314b88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:23.542457 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-bdc41b8d-3515-47cb-8920-761c8c9aa1b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.545902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-bdc41b8d-3515-47cb-8920-761c8c9aa1b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.554482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1097/2177] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:15:23 2022] DELETE /identity/v3/projects/2e93ca84c7e4403f8d6e0cb1c1314b88 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:23.561832 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f8b6e-67e7-48a7-a675-9d5e885baf7a None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:23.563031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f8b6e-67e7-48a7-a675-9d5e885baf7a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:23.563549 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f8b6e-67e7-48a7-a675-9d5e885baf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:23.595481 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-dc2f8b6e-67e7-48a7-a675-9d5e885baf7a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:23.597534 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-dc2f8b6e-67e7-48a7-a675-9d5e885baf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:23.856962 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1081/2178] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:23.876106 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e84d5a-9e49-4e23-8815-fb705080e1a0 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:23.876766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e84d5a-9e49-4e23-8815-fb705080e1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:23.877328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e84d5a-9e49-4e23-8815-fb705080e1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:23.920213 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-b4e84d5a-9e49-4e23-8815-fb705080e1a0 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:23.921966 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b4e84d5a-9e49-4e23-8815-fb705080e1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:23.948795 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1098/2179] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:23.955391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f8b6e-67e7-48a7-a675-9d5e885baf7a None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:23.985465 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f8b6e-67e7-48a7-a675-9d5e885baf7a None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:24.015378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d9bc7-16c4-4ddb-b5c1-c822ccb4f024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:24.016579 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d9bc7-16c4-4ddb-b5c1-c822ccb4f024 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:24.017163 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d9bc7-16c4-4ddb-b5c1-c822ccb4f024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:24.017730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d9bc7-16c4-4ddb-b5c1-c822ccb4f024 admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.019748 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d9bc7-16c4-4ddb-b5c1-c822ccb4f024 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:15:24.020712 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d9bc7-16c4-4ddb-b5c1-c822ccb4f024 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:24.026009 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d9bc7-16c4-4ddb-b5c1-c822ccb4f024 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:24.045593 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:24.078586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1082/2180] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:15:24.080456 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e84d5a-9e49-4e23-8815-fb705080e1a0 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:24.109810 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e84d5a-9e49-4e23-8815-fb705080e1a0 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:24.136339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec61d7-6326-40c3-a451-7e85ca1d95c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:24.138682 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec61d7-6326-40c3-a451-7e85ca1d95c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:24.139179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec61d7-6326-40c3-a451-7e85ca1d95c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:24.139698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec61d7-6326-40c3-a451-7e85ca1d95c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.141178 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec61d7-6326-40c3-a451-7e85ca1d95c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:24.149946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec61d7-6326-40c3-a451-7e85ca1d95c9 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:24.310005 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1099/2181] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:15:24 2022] POST /identity/v3/projects => generated 363 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:15:24.313719 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d9bc7-16c4-4ddb-b5c1-c822ccb4f024 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:24.359281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d9bc7-16c4-4ddb-b5c1-c822ccb4f024 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:24.624855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f3f25-9c03-41c7-91ac-a2e4666fc113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:24.626246 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f3f25-9c03-41c7-91ac-a2e4666fc113 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:24.626606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f3f25-9c03-41c7-91ac-a2e4666fc113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:24.627031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f3f25-9c03-41c7-91ac-a2e4666fc113 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.627906 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f3f25-9c03-41c7-91ac-a2e4666fc113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:24.630861 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f3f25-9c03-41c7-91ac-a2e4666fc113 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:24.676256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1083/2182] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:15:24 2022] POST /identity/v3/users => generated 282 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:24.682228 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec61d7-6326-40c3-a451-7e85ca1d95c9 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:24.711800 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec61d7-6326-40c3-a451-7e85ca1d95c9 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:24.742790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421923c9-1e54-4e60-b18e-acbdb1879235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:24.744264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421923c9-1e54-4e60-b18e-acbdb1879235 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:24.744888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421923c9-1e54-4e60-b18e-acbdb1879235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:24.745334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421923c9-1e54-4e60-b18e-acbdb1879235 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.746474 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421923c9-1e54-4e60-b18e-acbdb1879235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:24.749629 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421923c9-1e54-4e60-b18e-acbdb1879235 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:24.855049 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1100/2183] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:15:24.879492 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f3f25-9c03-41c7-91ac-a2e4666fc113 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:24.937562 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f3f25-9c03-41c7-91ac-a2e4666fc113 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:24.958832 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b15ff8-d087-4cb3-bab9-640573a80da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:24.960157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b15ff8-d087-4cb3-bab9-640573a80da1 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:24.960855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b15ff8-d087-4cb3-bab9-640573a80da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:24.961460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b15ff8-d087-4cb3-bab9-640573a80da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.962913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b15ff8-d087-4cb3-bab9-640573a80da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:24.966457 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b15ff8-d087-4cb3-bab9-640573a80da1 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:24.992666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1084/2184] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:24.999543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421923c9-1e54-4e60-b18e-acbdb1879235 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:25.022660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421923c9-1e54-4e60-b18e-acbdb1879235 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:25.030632 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39132f-3a4f-4914-bf3e-4d2ccb8a7d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:25.032347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39132f-3a4f-4914-bf3e-4d2ccb8a7d24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:25.032827 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39132f-3a4f-4914-bf3e-4d2ccb8a7d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:25.033105 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39132f-3a4f-4914-bf3e-4d2ccb8a7d24 admin admin] PATH_INFO: `/v3/projects/ebee9e5d0180489b828f9446a61829d7/users/32a6bb1ac9d34349929ced0b376d9031/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:25.059020 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39132f-3a4f-4914-bf3e-4d2ccb8a7d24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebee9e5d0180489b828f9446a61829d7, user_id=32a6bb1ac9d34349929ced0b376d9031, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:25.064148 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39132f-3a4f-4914-bf3e-4d2ccb8a7d24 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:25.098042 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1101/2185] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:15:24 2022] PUT /identity/v3/projects/ebee9e5d0180489b828f9446a61829d7/users/32a6bb1ac9d34349929ced0b376d9031/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:25.110060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6808f-cc88-4fcd-8f33-d6ecc762a0bf None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:25.110363 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6808f-cc88-4fcd-8f33-d6ecc762a0bf None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:25.110604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6808f-cc88-4fcd-8f33-d6ecc762a0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:25.161374 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-9bb6808f-cc88-4fcd-8f33-d6ecc762a0bf None None] MFA Rules not processed for user `32a6bb1ac9d34349929ced0b376d9031`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:25.163056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-9bb6808f-cc88-4fcd-8f33-d6ecc762a0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:25.300589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1085/2186] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:25 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:25.305137 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39132f-3a4f-4914-bf3e-4d2ccb8a7d24 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:25.431813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39132f-3a4f-4914-bf3e-4d2ccb8a7d24 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:25.439614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84137f88-b8c7-472d-ab02-7967432325d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:25.440881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84137f88-b8c7-472d-ab02-7967432325d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:25.441104 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84137f88-b8c7-472d-ab02-7967432325d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:25.441550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84137f88-b8c7-472d-ab02-7967432325d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:25.442763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84137f88-b8c7-472d-ab02-7967432325d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:25.447010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84137f88-b8c7-472d-ab02-7967432325d5 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:25.498898 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1102/2187] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:15:25 2022] POST /identity/v3/projects => generated 361 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:15:25.500748 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6808f-cc88-4fcd-8f33-d6ecc762a0bf None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:25.518023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6808f-cc88-4fcd-8f33-d6ecc762a0bf None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:25.629516 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b38ae-0937-4b7a-9365-85a00f0ced31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:25.630591 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b38ae-0937-4b7a-9365-85a00f0ced31 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:25.631677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b38ae-0937-4b7a-9365-85a00f0ced31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:25.632087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b38ae-0937-4b7a-9365-85a00f0ced31 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:25.633404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b38ae-0937-4b7a-9365-85a00f0ced31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:25.637323 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b38ae-0937-4b7a-9365-85a00f0ced31 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:25.656275 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:25.671808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1086/2188] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:15:25 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:25.677020 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84137f88-b8c7-472d-ab02-7967432325d5 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:25.688994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84137f88-b8c7-472d-ab02-7967432325d5 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:25.694483 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fc62d-0799-4077-a25b-c44956eab848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:25.695132 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fc62d-0799-4077-a25b-c44956eab848 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:25.695416 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fc62d-0799-4077-a25b-c44956eab848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:25.695696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fc62d-0799-4077-a25b-c44956eab848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:25.696418 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fc62d-0799-4077-a25b-c44956eab848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:25.698150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fc62d-0799-4077-a25b-c44956eab848 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:25.713994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1103/2189] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:25.721440 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b38ae-0937-4b7a-9365-85a00f0ced31 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:25.734664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b38ae-0937-4b7a-9365-85a00f0ced31 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:25.740177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c87262-5a3b-402a-b762-95bd37bc286a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:25.740973 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c87262-5a3b-402a-b762-95bd37bc286a admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:25.741247 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c87262-5a3b-402a-b762-95bd37bc286a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:25.741521 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c87262-5a3b-402a-b762-95bd37bc286a admin admin] PATH_INFO: `/v3/projects/0d524124c8b940aeb2812d9f2f15b94d/users/e77ebececbb04f248b0e9b4b3130af9d/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:25.763265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c87262-5a3b-402a-b762-95bd37bc286a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d524124c8b940aeb2812d9f2f15b94d, user_id=e77ebececbb04f248b0e9b4b3130af9d, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:25.766198 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c87262-5a3b-402a-b762-95bd37bc286a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:25.785875 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1087/2190] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:15:25 2022] PUT /identity/v3/projects/0d524124c8b940aeb2812d9f2f15b94d/users/e77ebececbb04f248b0e9b4b3130af9d/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:25.786130 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fc62d-0799-4077-a25b-c44956eab848 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:25.789553 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0d3fc62d-0799-4077-a25b-c44956eab848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:25.791925 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c87262-5a3b-402a-b762-95bd37bc286a admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:25.814824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fc62d-0799-4077-a25b-c44956eab848 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:25.878290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c87262-5a3b-402a-b762-95bd37bc286a admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:25.884601 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e16c7-f8e3-47bb-8cec-1f7b2421f465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:25.885397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e16c7-f8e3-47bb-8cec-1f7b2421f465 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:25.885816 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e16c7-f8e3-47bb-8cec-1f7b2421f465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:25.886218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e16c7-f8e3-47bb-8cec-1f7b2421f465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:25.887060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e16c7-f8e3-47bb-8cec-1f7b2421f465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:25.889017 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e16c7-f8e3-47bb-8cec-1f7b2421f465 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:25.903379 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1088/2191] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:15:25.908433 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e16c7-f8e3-47bb-8cec-1f7b2421f465 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:25.922874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e16c7-f8e3-47bb-8cec-1f7b2421f465 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:25.926105 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948bea6-761f-44e6-9ca8-3db3b70fa827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:25.927157 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948bea6-761f-44e6-9ca8-3db3b70fa827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:25.927618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948bea6-761f-44e6-9ca8-3db3b70fa827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:25.928199 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948bea6-761f-44e6-9ca8-3db3b70fa827 admin admin] PATH_INFO: `/v3/users/e47d2e62189b477f93e046b844195461` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:25.930238 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e948bea6-761f-44e6-9ca8-3db3b70fa827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e47d2e62189b477f93e046b844195461)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:25.930782 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860bb09-863f-4797-8240-24fccd24dc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:25.931843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860bb09-863f-4797-8240-24fccd24dc8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:25.932292 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860bb09-863f-4797-8240-24fccd24dc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:25.932815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860bb09-863f-4797-8240-24fccd24dc8a admin admin] PATH_INFO: `/v3/projects/0d524124c8b940aeb2812d9f2f15b94d/users/e77ebececbb04f248b0e9b4b3130af9d/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:25.933890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e948bea6-761f-44e6-9ca8-3db3b70fa827 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:25.936450 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2860bb09-863f-4797-8240-24fccd24dc8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d524124c8b940aeb2812d9f2f15b94d, user_id=e77ebececbb04f248b0e9b4b3130af9d, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:25.941196 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2860bb09-863f-4797-8240-24fccd24dc8a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:25.962833 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1089/2192] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:15:25 2022] PUT /identity/v3/projects/0d524124c8b940aeb2812d9f2f15b94d/users/e77ebececbb04f248b0e9b4b3130af9d/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:25.963394 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:25.993157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d20aca-5763-4d46-a6b3-4c5ac73ba295 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:25.993632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d20aca-5763-4d46-a6b3-4c5ac73ba295 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:25.994045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d20aca-5763-4d46-a6b3-4c5ac73ba295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.053682 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-a7d20aca-5763-4d46-a6b3-4c5ac73ba295 None None] MFA Rules not processed for user `e77ebececbb04f248b0e9b4b3130af9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:26.056170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a7d20aca-5763-4d46-a6b3-4c5ac73ba295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:26.093324 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e948bea6-761f-44e6-9ca8-3db3b70fa827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e47d2e62189b477f93e046b844195461'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:26.104321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e948bea6-761f-44e6-9ca8-3db3b70fa827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e47d2e62189b477f93e046b844195461'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:26.123516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e948bea6-761f-44e6-9ca8-3db3b70fa827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e47d2e62189b477f93e046b844195461'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:26.124787 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e948bea6-761f-44e6-9ca8-3db3b70fa827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e47d2e62189b477f93e046b844195461'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:26.126001 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e948bea6-761f-44e6-9ca8-3db3b70fa827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e47d2e62189b477f93e046b844195461'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:26.170691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1104/2193] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:15:25 2022] DELETE /identity/v3/users/e47d2e62189b477f93e046b844195461 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:26.177405 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9494bf-8aca-40fe-8acf-9bdc6662d5df None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.177949 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9494bf-8aca-40fe-8acf-9bdc6662d5df None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.178441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9494bf-8aca-40fe-8acf-9bdc6662d5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.209504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-ec9494bf-8aca-40fe-8acf-9bdc6662d5df None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:26.211006 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ec9494bf-8aca-40fe-8acf-9bdc6662d5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:26.228307 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1090/2194] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:25 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:26.246255 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d20aca-5763-4d46-a6b3-4c5ac73ba295 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.248641 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a7d20aca-5763-4d46-a6b3-4c5ac73ba295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:26.309536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1105/2195] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:26.322780 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9494bf-8aca-40fe-8acf-9bdc6662d5df None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.324823 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ec9494bf-8aca-40fe-8acf-9bdc6662d5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:26.330603 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:26.354351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d20aca-5763-4d46-a6b3-4c5ac73ba295 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.403675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9494bf-8aca-40fe-8acf-9bdc6662d5df None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.416779 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61835b-fa1a-4eb2-b475-ce8dbf2f4236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.417853 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61835b-fa1a-4eb2-b475-ce8dbf2f4236 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.418162 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61835b-fa1a-4eb2-b475-ce8dbf2f4236 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.418462 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61835b-fa1a-4eb2-b475-ce8dbf2f4236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.425740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222fe45-ceed-4853-91c4-daa2063f9221 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.426341 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222fe45-ceed-4853-91c4-daa2063f9221 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.426728 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222fe45-ceed-4853-91c4-daa2063f9221 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.426882 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222fe45-ceed-4853-91c4-daa2063f9221 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.439035 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61835b-fa1a-4eb2-b475-ce8dbf2f4236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.444047 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61835b-fa1a-4eb2-b475-ce8dbf2f4236 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.468065 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1106/2196] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:15:26.523499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2222fe45-ceed-4853-91c4-daa2063f9221 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.527853 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2222fe45-ceed-4853-91c4-daa2063f9221 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.545684 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1091/2197] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:26 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:26.708714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61835b-fa1a-4eb2-b475-ce8dbf2f4236 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.722149 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61835b-fa1a-4eb2-b475-ce8dbf2f4236 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.729827 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c0f01-3679-41ea-a956-a20e130d10ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.731173 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c0f01-3679-41ea-a956-a20e130d10ee service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.731765 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c0f01-3679-41ea-a956-a20e130d10ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.732665 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c0f01-3679-41ea-a956-a20e130d10ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.735994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-172c0f01-3679-41ea-a956-a20e130d10ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:26.787402 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:26.855856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c0f01-3679-41ea-a956-a20e130d10ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.857909 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c0f01-3679-41ea-a956-a20e130d10ee service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.871128 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1107/2198] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:26 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:15:27.039083 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222fe45-ceed-4853-91c4-daa2063f9221 service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:27.042330 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-2222fe45-ceed-4853-91c4-daa2063f9221 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:27.070013 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222fe45-ceed-4853-91c4-daa2063f9221 service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:27.080229 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e883583-cc16-4ac9-b0e5-72c1f5b746a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:27.081776 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e883583-cc16-4ac9-b0e5-72c1f5b746a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:27.082671 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e883583-cc16-4ac9-b0e5-72c1f5b746a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:27.083473 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e883583-cc16-4ac9-b0e5-72c1f5b746a1 admin admin] PATH_INFO: `/v3/projects/bac59c62cb6c4943b193c9f586c29c8f` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:27.086806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e883583-cc16-4ac9-b0e5-72c1f5b746a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bac59c62cb6c4943b193c9f586c29c8f)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:27.091309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e883583-cc16-4ac9-b0e5-72c1f5b746a1 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:27.182160 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-8e883583-cc16-4ac9-b0e5-72c1f5b746a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bac59c62cb6c4943b193c9f586c29c8f', 'request_id': 'req-8e883583-cc16-4ac9-b0e5-72c1f5b746a1'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:27.195159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-8e883583-cc16-4ac9-b0e5-72c1f5b746a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bac59c62cb6c4943b193c9f586c29c8f', 'request_id': 'req-8e883583-cc16-4ac9-b0e5-72c1f5b746a1'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:27.199742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-8e883583-cc16-4ac9-b0e5-72c1f5b746a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bac59c62cb6c4943b193c9f586c29c8f', 'request_id': 'req-8e883583-cc16-4ac9-b0e5-72c1f5b746a1'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:27.215745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-8e883583-cc16-4ac9-b0e5-72c1f5b746a1 admin admin] The token cache is being invalidate because project bac59c62cb6c4943b193c9f586c29c8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:27.216032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-8e883583-cc16-4ac9-b0e5-72c1f5b746a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:27.216801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-8e883583-cc16-4ac9-b0e5-72c1f5b746a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:27.218389 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1092/2199] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:15:27 2022] DELETE /identity/v3/projects/bac59c62cb6c4943b193c9f586c29c8f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:27.224256 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597ccef-6b0f-497b-baff-0e074b1efec8 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:27.224976 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597ccef-6b0f-497b-baff-0e074b1efec8 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:27.224976 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597ccef-6b0f-497b-baff-0e074b1efec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:27.248027 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-6597ccef-6b0f-497b-baff-0e074b1efec8 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:27.249392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-6597ccef-6b0f-497b-baff-0e074b1efec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:27.324677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1108/2200] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:27.333599 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2e9cd-2de6-4620-856b-4dc679e2bcc0 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:27.333912 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2e9cd-2de6-4620-856b-4dc679e2bcc0 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:27.334681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2e9cd-2de6-4620-856b-4dc679e2bcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:27.359184 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-18d2e9cd-2de6-4620-856b-4dc679e2bcc0 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:27.360807 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-18d2e9cd-2de6-4620-856b-4dc679e2bcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:27.377669 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1093/2201] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:27.380237 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597ccef-6b0f-497b-baff-0e074b1efec8 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:27.398545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597ccef-6b0f-497b-baff-0e074b1efec8 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:27.406936 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90db9ed-3216-4183-bde5-90ded1e77da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:27.408029 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90db9ed-3216-4183-bde5-90ded1e77da1 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:27.408566 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90db9ed-3216-4183-bde5-90ded1e77da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:27.409067 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90db9ed-3216-4183-bde5-90ded1e77da1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:27.410481 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90db9ed-3216-4183-bde5-90ded1e77da1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:15:27.411045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90db9ed-3216-4183-bde5-90ded1e77da1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:27.415122 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90db9ed-3216-4183-bde5-90ded1e77da1 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:27.436181 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1109/2202] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:15:27.441862 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2e9cd-2de6-4620-856b-4dc679e2bcc0 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:27.457162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2e9cd-2de6-4620-856b-4dc679e2bcc0 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:27.463581 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0eca05-c02a-4457-963c-85db945693bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:27.464647 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0eca05-c02a-4457-963c-85db945693bd admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:27.465055 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0eca05-c02a-4457-963c-85db945693bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:27.465378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0eca05-c02a-4457-963c-85db945693bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:27.466134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0eca05-c02a-4457-963c-85db945693bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:27.468392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0eca05-c02a-4457-963c-85db945693bd admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:27.502906 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:27.516325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1094/2203] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:15:27 2022] POST /identity/v3/projects => generated 397 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:27.522433 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90db9ed-3216-4183-bde5-90ded1e77da1 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:27.537784 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90db9ed-3216-4183-bde5-90ded1e77da1 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:27.615396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be2cac-6548-458c-a003-807255e75579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:27.615914 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be2cac-6548-458c-a003-807255e75579 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:27.616232 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be2cac-6548-458c-a003-807255e75579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:27.616536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be2cac-6548-458c-a003-807255e75579 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:27.617225 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be2cac-6548-458c-a003-807255e75579 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:27.619280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be2cac-6548-458c-a003-807255e75579 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:27.651929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1110/2204] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:15:27 2022] POST /identity/v3/users => generated 299 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:27.658134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0eca05-c02a-4457-963c-85db945693bd admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:27.675929 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0eca05-c02a-4457-963c-85db945693bd admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:27.684733 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17482c23-3ef5-4d6a-880c-f6e2d31bd691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:27.685724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17482c23-3ef5-4d6a-880c-f6e2d31bd691 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:27.687597 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17482c23-3ef5-4d6a-880c-f6e2d31bd691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:27.687597 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17482c23-3ef5-4d6a-880c-f6e2d31bd691 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:27.687926 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17482c23-3ef5-4d6a-880c-f6e2d31bd691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:27.690837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17482c23-3ef5-4d6a-880c-f6e2d31bd691 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:27.714548 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1095/2205] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:27.721945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be2cac-6548-458c-a003-807255e75579 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:27.743510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be2cac-6548-458c-a003-807255e75579 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:27.748962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c31ed-a70a-40c4-b718-d7654e536f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:27.749605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c31ed-a70a-40c4-b718-d7654e536f6d admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:27.749881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c31ed-a70a-40c4-b718-d7654e536f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:27.750154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c31ed-a70a-40c4-b718-d7654e536f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:27.750864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676c31ed-a70a-40c4-b718-d7654e536f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:27.752541 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676c31ed-a70a-40c4-b718-d7654e536f6d admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:27.772756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1111/2206] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:27.774122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17482c23-3ef5-4d6a-880c-f6e2d31bd691 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:27.789899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17482c23-3ef5-4d6a-880c-f6e2d31bd691 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:27.797870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9a5b0-ca13-4410-93b4-c692c1a7e32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:27.799037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9a5b0-ca13-4410-93b4-c692c1a7e32c admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:27.799540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9a5b0-ca13-4410-93b4-c692c1a7e32c admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:27.800025 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9a5b0-ca13-4410-93b4-c692c1a7e32c admin admin] PATH_INFO: `/v3/projects/c4a43813160547138dbb6585cf658d1d/users/bd055ef12c0649128adf23e5a25681ba/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:27.821840 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9a5b0-ca13-4410-93b4-c692c1a7e32c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4a43813160547138dbb6585cf658d1d, user_id=bd055ef12c0649128adf23e5a25681ba, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:27.825070 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9a5b0-ca13-4410-93b4-c692c1a7e32c admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:27.847647 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1096/2207] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:15:27 2022] PUT /identity/v3/projects/c4a43813160547138dbb6585cf658d1d/users/bd055ef12c0649128adf23e5a25681ba/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:27.935752 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757f895-5940-4aa8-8716-821c29266e19 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:27.936354 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757f895-5940-4aa8-8716-821c29266e19 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:27.936870 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757f895-5940-4aa8-8716-821c29266e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:27.985740 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-3757f895-5940-4aa8-8716-821c29266e19 None None] MFA Rules not processed for user `bd055ef12c0649128adf23e5a25681ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:27.989860 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3757f895-5940-4aa8-8716-821c29266e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:28.058318 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:28.131900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1112/2208] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:28.148287 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9a5b0-ca13-4410-93b4-c692c1a7e32c admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:28.150836 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d3c9a5b0-ca13-4410-93b4-c692c1a7e32c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:28.253996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9a5b0-ca13-4410-93b4-c692c1a7e32c admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:28.261379 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63534f12-cb76-43fa-a23a-fc30d791e938 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:28.263363 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63534f12-cb76-43fa-a23a-fc30d791e938 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:28.271911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63534f12-cb76-43fa-a23a-fc30d791e938 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:28.273485 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63534f12-cb76-43fa-a23a-fc30d791e938 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:28.298436 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63534f12-cb76-43fa-a23a-fc30d791e938 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:28.301478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63534f12-cb76-43fa-a23a-fc30d791e938 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:28.356989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1097/2209] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:38.127329 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=15 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:38.331388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:39.027936 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e82224-1e25-470d-bc6d-52ca01d0a079 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:39.028286 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e82224-1e25-470d-bc6d-52ca01d0a079 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:39.028603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e82224-1e25-470d-bc6d-52ca01d0a079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:39.050911 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-48e82224-1e25-470d-bc6d-52ca01d0a079 None None] MFA Rules not processed for user `4370f8f474c2446e989d0d8c5b75e87a`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:39.052302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-48e82224-1e25-470d-bc6d-52ca01d0a079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:39.220507 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1113/2210] 10.222.0.93 () {58 vars in 1009 bytes} [Mon Aug 1 15:15:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:39.290703 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63534f12-cb76-43fa-a23a-fc30d791e938 service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:39.293007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-63534f12-cb76-43fa-a23a-fc30d791e938 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:39.488446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63534f12-cb76-43fa-a23a-fc30d791e938 service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:39.509800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68435878-3720-46d1-b15b-a4a11c2719df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4df37b02152c48509243b73ebbf49d8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:39.510471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68435878-3720-46d1-b15b-a4a11c2719df service swift] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:39.510757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68435878-3720-46d1-b15b-a4a11c2719df service swift] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:39.511087 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68435878-3720-46d1-b15b-a4a11c2719df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:39.543942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68435878-3720-46d1-b15b-a4a11c2719df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:39.553407 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68435878-3720-46d1-b15b-a4a11c2719df service swift] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:39.602040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1098/2211] 10.222.0.93 () {60 vars in 1384 bytes} [Mon Aug 1 15:15:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:15:39.625498 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e82224-1e25-470d-bc6d-52ca01d0a079 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:39.987103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e82224-1e25-470d-bc6d-52ca01d0a079 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:39.999450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd28dd-7980-4572-a3a2-b5f148c79e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:40.000853 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd28dd-7980-4572-a3a2-b5f148c79e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:40.001345 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd28dd-7980-4572-a3a2-b5f148c79e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:40.001830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd28dd-7980-4572-a3a2-b5f148c79e2e admin admin] PATH_INFO: `/v3/users/bd055ef12c0649128adf23e5a25681ba` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:40.004856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd28dd-7980-4572-a3a2-b5f148c79e2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd055ef12c0649128adf23e5a25681ba)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:40.010762 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd28dd-7980-4572-a3a2-b5f148c79e2e admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:40.173814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a8dd28dd-7980-4572-a3a2-b5f148c79e2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd055ef12c0649128adf23e5a25681ba'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:40.191692 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a8dd28dd-7980-4572-a3a2-b5f148c79e2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd055ef12c0649128adf23e5a25681ba'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:40.216906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a8dd28dd-7980-4572-a3a2-b5f148c79e2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd055ef12c0649128adf23e5a25681ba'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:40.221057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a8dd28dd-7980-4572-a3a2-b5f148c79e2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd055ef12c0649128adf23e5a25681ba'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:40.226786 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a8dd28dd-7980-4572-a3a2-b5f148c79e2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd055ef12c0649128adf23e5a25681ba'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:40.287735 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1114/2212] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:15:39 2022] DELETE /identity/v3/users/bd055ef12c0649128adf23e5a25681ba => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:40.294793 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ff1a4-d3de-40e4-9903-8e7e3f0b8029 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:40.295828 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ff1a4-d3de-40e4-9903-8e7e3f0b8029 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:40.297254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ff1a4-d3de-40e4-9903-8e7e3f0b8029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:40.437030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-480ff1a4-d3de-40e4-9903-8e7e3f0b8029 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:40.439058 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-480ff1a4-d3de-40e4-9903-8e7e3f0b8029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:40.831849 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1099/2213] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 540 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:40.848257 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd28dd-7980-4572-a3a2-b5f148c79e2e admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:40.851380 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a8dd28dd-7980-4572-a3a2-b5f148c79e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:40.908201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:41.040907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd28dd-7980-4572-a3a2-b5f148c79e2e admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:41.047575 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa4a31-0655-44c9-ae81-495c06165653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:41.049251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa4a31-0655-44c9-ae81-495c06165653 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:41.049984 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa4a31-0655-44c9-ae81-495c06165653 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:41.050678 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa4a31-0655-44c9-ae81-495c06165653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:41.089962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa4a31-0655-44c9-ae81-495c06165653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:41.092912 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa4a31-0655-44c9-ae81-495c06165653 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:41.146291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1115/2214] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:42.323864 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ff1a4-d3de-40e4-9903-8e7e3f0b8029 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:42.325883 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-480ff1a4-d3de-40e4-9903-8e7e3f0b8029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:42.365301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ff1a4-d3de-40e4-9903-8e7e3f0b8029 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:42.387365 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979276ee-50a4-4ba8-8a13-877e4c21ba49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:42.388370 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979276ee-50a4-4ba8-8a13-877e4c21ba49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:42.389016 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979276ee-50a4-4ba8-8a13-877e4c21ba49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:42.389454 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979276ee-50a4-4ba8-8a13-877e4c21ba49 admin admin] PATH_INFO: `/v3/projects/c4a43813160547138dbb6585cf658d1d` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:42.393133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979276ee-50a4-4ba8-8a13-877e4c21ba49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4a43813160547138dbb6585cf658d1d)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:42.405366 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979276ee-50a4-4ba8-8a13-877e4c21ba49 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:42.607561 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-979276ee-50a4-4ba8-8a13-877e4c21ba49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4a43813160547138dbb6585cf658d1d', 'request_id': 'req-979276ee-50a4-4ba8-8a13-877e4c21ba49'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:42.631578 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-979276ee-50a4-4ba8-8a13-877e4c21ba49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4a43813160547138dbb6585cf658d1d', 'request_id': 'req-979276ee-50a4-4ba8-8a13-877e4c21ba49'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:42.662952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-979276ee-50a4-4ba8-8a13-877e4c21ba49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4a43813160547138dbb6585cf658d1d', 'request_id': 'req-979276ee-50a4-4ba8-8a13-877e4c21ba49'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:42.716850 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-979276ee-50a4-4ba8-8a13-877e4c21ba49 admin admin] The token cache is being invalidate because project c4a43813160547138dbb6585cf658d1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:42.717350 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-979276ee-50a4-4ba8-8a13-877e4c21ba49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:42.721442 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-979276ee-50a4-4ba8-8a13-877e4c21ba49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:42.736488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1100/2215] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:15:42 2022] DELETE /identity/v3/projects/c4a43813160547138dbb6585cf658d1d => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:44.003581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa4a31-0655-44c9-ae81-495c06165653 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:44.006532 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-c3fa4a31-0655-44c9-ae81-495c06165653 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:44.055330 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa4a31-0655-44c9-ae81-495c06165653 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:44.326481 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760ad4a-1ed0-453e-9e78-f59259f6ca28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:44.327623 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760ad4a-1ed0-453e-9e78-f59259f6ca28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:44.328274 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760ad4a-1ed0-453e-9e78-f59259f6ca28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:44.344716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760ad4a-1ed0-453e-9e78-f59259f6ca28 admin admin] PATH_INFO: `/v3/users/32a6bb1ac9d34349929ced0b376d9031` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:44.347327 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760ad4a-1ed0-453e-9e78-f59259f6ca28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32a6bb1ac9d34349929ced0b376d9031)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:44.351221 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760ad4a-1ed0-453e-9e78-f59259f6ca28 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:44.612102 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d760ad4a-1ed0-453e-9e78-f59259f6ca28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32a6bb1ac9d34349929ced0b376d9031'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:44.638679 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d760ad4a-1ed0-453e-9e78-f59259f6ca28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32a6bb1ac9d34349929ced0b376d9031'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:44.745659 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d760ad4a-1ed0-453e-9e78-f59259f6ca28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32a6bb1ac9d34349929ced0b376d9031'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:44.746476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d760ad4a-1ed0-453e-9e78-f59259f6ca28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32a6bb1ac9d34349929ced0b376d9031'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:44.747253 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d760ad4a-1ed0-453e-9e78-f59259f6ca28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32a6bb1ac9d34349929ced0b376d9031'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:44.786538 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1116/2216] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:15:43 2022] DELETE /identity/v3/users/32a6bb1ac9d34349929ced0b376d9031 => generated 0 bytes in 786 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:44.807017 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebba91-385a-483d-9b7d-df6e24562c21 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:44.807965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebba91-385a-483d-9b7d-df6e24562c21 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:44.807965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebba91-385a-483d-9b7d-df6e24562c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:44.867705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-5eebba91-385a-483d-9b7d-df6e24562c21 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:44.870732 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-5eebba91-385a-483d-9b7d-df6e24562c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:44.889760 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:44.979168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1101/2217] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:44.992252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760ad4a-1ed0-453e-9e78-f59259f6ca28 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:44.994628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-d760ad4a-1ed0-453e-9e78-f59259f6ca28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:45.102631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:45.145816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760ad4a-1ed0-453e-9e78-f59259f6ca28 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:45.183044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f8947-5a13-43d6-b5eb-3f0ff8e44d87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:45.184716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f8947-5a13-43d6-b5eb-3f0ff8e44d87 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:45.185220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f8947-5a13-43d6-b5eb-3f0ff8e44d87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:45.186037 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f8947-5a13-43d6-b5eb-3f0ff8e44d87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:45.214599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f8947-5a13-43d6-b5eb-3f0ff8e44d87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:45.217944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f8947-5a13-43d6-b5eb-3f0ff8e44d87 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:45.240322 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1117/2218] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:45.940971 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebba91-385a-483d-9b7d-df6e24562c21 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:45.945486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-5eebba91-385a-483d-9b7d-df6e24562c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:46.014729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebba91-385a-483d-9b7d-df6e24562c21 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:46.029602 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba96b3-a3a3-420f-a6ac-1f737b164b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:46.031059 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba96b3-a3a3-420f-a6ac-1f737b164b8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:46.031734 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba96b3-a3a3-420f-a6ac-1f737b164b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:46.032316 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba96b3-a3a3-420f-a6ac-1f737b164b8d admin admin] PATH_INFO: `/v3/projects/ebee9e5d0180489b828f9446a61829d7` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:46.036211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba96b3-a3a3-420f-a6ac-1f737b164b8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebee9e5d0180489b828f9446a61829d7)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:46.047416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba96b3-a3a3-420f-a6ac-1f737b164b8d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:46.352219 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-53ba96b3-a3a3-420f-a6ac-1f737b164b8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebee9e5d0180489b828f9446a61829d7', 'request_id': 'req-53ba96b3-a3a3-420f-a6ac-1f737b164b8d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:46.401116 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-53ba96b3-a3a3-420f-a6ac-1f737b164b8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebee9e5d0180489b828f9446a61829d7', 'request_id': 'req-53ba96b3-a3a3-420f-a6ac-1f737b164b8d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:46.431452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-53ba96b3-a3a3-420f-a6ac-1f737b164b8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebee9e5d0180489b828f9446a61829d7', 'request_id': 'req-53ba96b3-a3a3-420f-a6ac-1f737b164b8d'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:46.469899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-53ba96b3-a3a3-420f-a6ac-1f737b164b8d admin admin] The token cache is being invalidate because project ebee9e5d0180489b828f9446a61829d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:46.472154 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-53ba96b3-a3a3-420f-a6ac-1f737b164b8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:46.485122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-53ba96b3-a3a3-420f-a6ac-1f737b164b8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:46.500286 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1102/2219] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:15:45 2022] DELETE /identity/v3/projects/ebee9e5d0180489b828f9446a61829d7 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:46.500983 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f8947-5a13-43d6-b5eb-3f0ff8e44d87 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:46.527943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f8947-5a13-43d6-b5eb-3f0ff8e44d87 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:46.809851 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ea6b5-ca57-4bef-b73d-e73f6e2023e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:46.811637 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ea6b5-ca57-4bef-b73d-e73f6e2023e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:46.812593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ea6b5-ca57-4bef-b73d-e73f6e2023e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:46.813514 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ea6b5-ca57-4bef-b73d-e73f6e2023e4 admin admin] PATH_INFO: `/v3/users/e77ebececbb04f248b0e9b4b3130af9d` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:46.816426 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939ea6b5-ca57-4bef-b73d-e73f6e2023e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e77ebececbb04f248b0e9b4b3130af9d)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:46.821915 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939ea6b5-ca57-4bef-b73d-e73f6e2023e4 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:47.007763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-939ea6b5-ca57-4bef-b73d-e73f6e2023e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e77ebececbb04f248b0e9b4b3130af9d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:47.030948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-939ea6b5-ca57-4bef-b73d-e73f6e2023e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e77ebececbb04f248b0e9b4b3130af9d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:47.050298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-939ea6b5-ca57-4bef-b73d-e73f6e2023e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e77ebececbb04f248b0e9b4b3130af9d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:47.051230 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-939ea6b5-ca57-4bef-b73d-e73f6e2023e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e77ebececbb04f248b0e9b4b3130af9d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:47.052235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-939ea6b5-ca57-4bef-b73d-e73f6e2023e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e77ebececbb04f248b0e9b4b3130af9d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:47.160505 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1118/2220] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:15:46 2022] DELETE /identity/v3/users/e77ebececbb04f248b0e9b4b3130af9d => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:48.019741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba96b3-a3a3-420f-a6ac-1f737b164b8d admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:48.023201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-53ba96b3-a3a3-420f-a6ac-1f737b164b8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:48.097498 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba96b3-a3a3-420f-a6ac-1f737b164b8d admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:48.426300 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7706c-8dcd-41b3-a860-6a41ea62c772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:48.427978 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7706c-8dcd-41b3-a860-6a41ea62c772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:48.428498 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7706c-8dcd-41b3-a860-6a41ea62c772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:48.428992 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7706c-8dcd-41b3-a860-6a41ea62c772 admin admin] PATH_INFO: `/v3/projects/0d524124c8b940aeb2812d9f2f15b94d` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:48.442337 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7706c-8dcd-41b3-a860-6a41ea62c772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d524124c8b940aeb2812d9f2f15b94d)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:48.446610 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7706c-8dcd-41b3-a860-6a41ea62c772 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:48.648760 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:48.682991 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-5db7706c-8dcd-41b3-a860-6a41ea62c772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d524124c8b940aeb2812d9f2f15b94d', 'request_id': 'req-5db7706c-8dcd-41b3-a860-6a41ea62c772'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:48.707655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-5db7706c-8dcd-41b3-a860-6a41ea62c772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d524124c8b940aeb2812d9f2f15b94d', 'request_id': 'req-5db7706c-8dcd-41b3-a860-6a41ea62c772'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:48.716528 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-5db7706c-8dcd-41b3-a860-6a41ea62c772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d524124c8b940aeb2812d9f2f15b94d', 'request_id': 'req-5db7706c-8dcd-41b3-a860-6a41ea62c772'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:48.750204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-5db7706c-8dcd-41b3-a860-6a41ea62c772 admin admin] The token cache is being invalidate because project 0d524124c8b940aeb2812d9f2f15b94d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:48.752137 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-5db7706c-8dcd-41b3-a860-6a41ea62c772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:48.753751 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-5db7706c-8dcd-41b3-a860-6a41ea62c772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:48.758359 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1103/2221] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:15:48 2022] DELETE /identity/v3/projects/0d524124c8b940aeb2812d9f2f15b94d => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:48.770208 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34262acb-9ec5-4fe5-8f8e-01832e47a73d None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:48.771868 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34262acb-9ec5-4fe5-8f8e-01832e47a73d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:48.771988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34262acb-9ec5-4fe5-8f8e-01832e47a73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:48.850917 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-34262acb-9ec5-4fe5-8f8e-01832e47a73d None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:48.853294 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-34262acb-9ec5-4fe5-8f8e-01832e47a73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:49.028371 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1119/2222] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:49.039496 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de499c-316d-425f-ac90-0efb4a2c412a None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:49.041533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de499c-316d-425f-ac90-0efb4a2c412a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:49.042042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de499c-316d-425f-ac90-0efb4a2c412a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:49.102656 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-f7de499c-316d-425f-ac90-0efb4a2c412a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:49.105717 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-f7de499c-316d-425f-ac90-0efb4a2c412a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:49.119025 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1104/2223] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:49.125458 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34262acb-9ec5-4fe5-8f8e-01832e47a73d None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:49.146780 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34262acb-9ec5-4fe5-8f8e-01832e47a73d None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:49.157611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcad87-4b1d-4616-9016-d3ee476e8934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:49.159295 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcad87-4b1d-4616-9016-d3ee476e8934 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:49.159681 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcad87-4b1d-4616-9016-d3ee476e8934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:49.160056 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcad87-4b1d-4616-9016-d3ee476e8934 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:49.161093 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcad87-4b1d-4616-9016-d3ee476e8934 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:15:49.161338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcad87-4b1d-4616-9016-d3ee476e8934 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:49.163553 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcad87-4b1d-4616-9016-d3ee476e8934 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:49.168495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:49.208506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de499c-316d-425f-ac90-0efb4a2c412a None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:49.217124 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1120/2224] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:15:49.260440 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de499c-316d-425f-ac90-0efb4a2c412a None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:49.291771 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0f2e9-2689-4741-a980-fced4bbd2dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:49.295473 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0f2e9-2689-4741-a980-fced4bbd2dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:49.299024 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0f2e9-2689-4741-a980-fced4bbd2dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:49.299701 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0f2e9-2689-4741-a980-fced4bbd2dc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:49.302917 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0f2e9-2689-4741-a980-fced4bbd2dc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:49.312732 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0f2e9-2689-4741-a980-fced4bbd2dc2 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:49.510827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1105/2225] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:15:49 2022] POST /identity/v3/projects => generated 369 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:49.517192 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcad87-4b1d-4616-9016-d3ee476e8934 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:49.545092 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcad87-4b1d-4616-9016-d3ee476e8934 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:49.798694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782cd6d-cae8-4e0a-aa2d-fa1a91208df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:49.801247 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782cd6d-cae8-4e0a-aa2d-fa1a91208df8 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:49.802598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782cd6d-cae8-4e0a-aa2d-fa1a91208df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:49.803654 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782cd6d-cae8-4e0a-aa2d-fa1a91208df8 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:49.805373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b782cd6d-cae8-4e0a-aa2d-fa1a91208df8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:49.814163 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b782cd6d-cae8-4e0a-aa2d-fa1a91208df8 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:49.906963 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1121/2226] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:15:49 2022] POST /identity/v3/users => generated 285 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:49.910441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0f2e9-2689-4741-a980-fced4bbd2dc2 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:49.980254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0f2e9-2689-4741-a980-fced4bbd2dc2 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:50.055360 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e50658-5e89-4104-916f-448373b6a392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:50.057262 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e50658-5e89-4104-916f-448373b6a392 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:50.062627 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e50658-5e89-4104-916f-448373b6a392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:50.065945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e50658-5e89-4104-916f-448373b6a392 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:50.069084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e50658-5e89-4104-916f-448373b6a392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:50.078903 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e50658-5e89-4104-916f-448373b6a392 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:50.180511 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1106/2227] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:50.194734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782cd6d-cae8-4e0a-aa2d-fa1a91208df8 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:50.229437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782cd6d-cae8-4e0a-aa2d-fa1a91208df8 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:50.257376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ab2e5-71c5-4312-88a9-847d42e653ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:50.258451 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ab2e5-71c5-4312-88a9-847d42e653ba admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:50.258875 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ab2e5-71c5-4312-88a9-847d42e653ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:50.259336 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ab2e5-71c5-4312-88a9-847d42e653ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:50.260512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ab2e5-71c5-4312-88a9-847d42e653ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:50.263470 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ab2e5-71c5-4312-88a9-847d42e653ba admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:50.305847 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1122/2228] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:50.312456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e50658-5e89-4104-916f-448373b6a392 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:50.352290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e50658-5e89-4104-916f-448373b6a392 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:50.394309 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae258f-1549-4352-a6c2-b3c7c9a8577a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:50.396300 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae258f-1549-4352-a6c2-b3c7c9a8577a admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:50.396819 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae258f-1549-4352-a6c2-b3c7c9a8577a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:50.397440 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae258f-1549-4352-a6c2-b3c7c9a8577a admin admin] PATH_INFO: `/v3/projects/2b34e6c52c70497286e957db51e06b21/users/02973bd442144ceb9b6bed4acca45c73/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:50.440956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae258f-1549-4352-a6c2-b3c7c9a8577a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b34e6c52c70497286e957db51e06b21, user_id=02973bd442144ceb9b6bed4acca45c73, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:50.447617 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae258f-1549-4352-a6c2-b3c7c9a8577a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:50.495708 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1107/2229] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:15:50 2022] PUT /identity/v3/projects/2b34e6c52c70497286e957db51e06b21/users/02973bd442144ceb9b6bed4acca45c73/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:50.513634 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9d35b-cd5f-4d2c-be6e-bae2ca8510b1 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:50.514143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9d35b-cd5f-4d2c-be6e-bae2ca8510b1 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:50.514679 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9d35b-cd5f-4d2c-be6e-bae2ca8510b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:50.620798 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-a3a9d35b-cd5f-4d2c-be6e-bae2ca8510b1 None None] MFA Rules not processed for user `02973bd442144ceb9b6bed4acca45c73`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:50.623143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a3a9d35b-cd5f-4d2c-be6e-bae2ca8510b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:50.876082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1123/2230] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:50 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:50.882398 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae258f-1549-4352-a6c2-b3c7c9a8577a admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:50.950360 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fe8ef-2df0-42aa-a789-e54e92a43be3 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:50.950651 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fe8ef-2df0-42aa-a789-e54e92a43be3 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:50.951083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fe8ef-2df0-42aa-a789-e54e92a43be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:51.015754 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-884fe8ef-2df0-42aa-a789-e54e92a43be3 None None] MFA Rules not processed for user `3af3a5c1c2f94e51bee936b4762dc14c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:51.019434 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-884fe8ef-2df0-42aa-a789-e54e92a43be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:51.134255 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae258f-1549-4352-a6c2-b3c7c9a8577a admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:51.154797 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b679c-6d7e-4674-aaa2-b01648c7d380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:51.156415 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b679c-6d7e-4674-aaa2-b01648c7d380 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:51.157218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b679c-6d7e-4674-aaa2-b01648c7d380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:51.158019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b679c-6d7e-4674-aaa2-b01648c7d380 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:51.159683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b679c-6d7e-4674-aaa2-b01648c7d380 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:51.165891 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b679c-6d7e-4674-aaa2-b01648c7d380 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:51.212749 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1124/2231] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:51.250418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1108/2232] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:15:50 2022] POST /identity/v3/projects => generated 367 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:51.256861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884fe8ef-2df0-42aa-a789-e54e92a43be3 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:51.278147 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884fe8ef-2df0-42aa-a789-e54e92a43be3 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:51.288359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:51.416139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb2035-afa5-4e40-9597-49e9aaf1623a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:51.417297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb2035-afa5-4e40-9597-49e9aaf1623a admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:51.417857 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb2035-afa5-4e40-9597-49e9aaf1623a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:51.418388 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb2035-afa5-4e40-9597-49e9aaf1623a admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:51.419804 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb2035-afa5-4e40-9597-49e9aaf1623a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:51.423934 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb2035-afa5-4e40-9597-49e9aaf1623a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:51.486311 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1125/2233] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:15:51 2022] POST /identity/v3/users => generated 290 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:15:51.490785 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b679c-6d7e-4674-aaa2-b01648c7d380 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:51.533497 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b679c-6d7e-4674-aaa2-b01648c7d380 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:51.547046 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca9ad4-af20-4441-b975-9127800381a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:51.548122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca9ad4-af20-4441-b975-9127800381a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:51.549057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca9ad4-af20-4441-b975-9127800381a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:51.553025 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca9ad4-af20-4441-b975-9127800381a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:51.566068 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca9ad4-af20-4441-b975-9127800381a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:51.568499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca9ad4-af20-4441-b975-9127800381a0 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:51.640674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1109/2234] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:15:51.642324 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb2035-afa5-4e40-9597-49e9aaf1623a admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:51.683779 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb2035-afa5-4e40-9597-49e9aaf1623a admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:51.698299 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3165f-2274-4651-8fec-5b89dcbe27cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:51.698299 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3165f-2274-4651-8fec-5b89dcbe27cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:51.698299 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3165f-2274-4651-8fec-5b89dcbe27cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:51.698299 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3165f-2274-4651-8fec-5b89dcbe27cb admin admin] PATH_INFO: `/v3/projects/8ba09803de5b4e52b4dc2993da623f05/users/725fae5c36ef41ab97e3d6c011400f3a/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:51.745835 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3165f-2274-4651-8fec-5b89dcbe27cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba09803de5b4e52b4dc2993da623f05, user_id=725fae5c36ef41ab97e3d6c011400f3a, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:51.757851 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3165f-2274-4651-8fec-5b89dcbe27cb admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:51.793207 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1126/2235] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:15:51 2022] PUT /identity/v3/projects/8ba09803de5b4e52b4dc2993da623f05/users/725fae5c36ef41ab97e3d6c011400f3a/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:51.798746 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca9ad4-af20-4441-b975-9127800381a0 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:51.815091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca9ad4-af20-4441-b975-9127800381a0 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:51.905454 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:51.993941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be069789-349e-4114-b8b7-b9da38c3da83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:51.994945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be069789-349e-4114-b8b7-b9da38c3da83 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:51.995315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be069789-349e-4114-b8b7-b9da38c3da83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:51.995619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be069789-349e-4114-b8b7-b9da38c3da83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:51.996488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be069789-349e-4114-b8b7-b9da38c3da83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:51.998291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be069789-349e-4114-b8b7-b9da38c3da83 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:52.058334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1110/2236] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:52.060059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3165f-2274-4651-8fec-5b89dcbe27cb admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:52.083657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3165f-2274-4651-8fec-5b89dcbe27cb admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:52.097332 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23162d0-c2ee-4a23-94dc-3f77ee1f5b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:52.099253 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23162d0-c2ee-4a23-94dc-3f77ee1f5b76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:52.099652 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23162d0-c2ee-4a23-94dc-3f77ee1f5b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:52.100047 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23162d0-c2ee-4a23-94dc-3f77ee1f5b76 admin admin] PATH_INFO: `/v3/projects/8ba09803de5b4e52b4dc2993da623f05/users/725fae5c36ef41ab97e3d6c011400f3a/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:52.104088 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23162d0-c2ee-4a23-94dc-3f77ee1f5b76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba09803de5b4e52b4dc2993da623f05, user_id=725fae5c36ef41ab97e3d6c011400f3a, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:52.119164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23162d0-c2ee-4a23-94dc-3f77ee1f5b76 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:52.237930 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1127/2237] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:15:52 2022] PUT /identity/v3/projects/8ba09803de5b4e52b4dc2993da623f05/users/725fae5c36ef41ab97e3d6c011400f3a/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:52.244976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0210d69-6134-48d0-a722-fcd5656e0977 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:52.245902 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0210d69-6134-48d0-a722-fcd5656e0977 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:52.246886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0210d69-6134-48d0-a722-fcd5656e0977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:52.307115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-d0210d69-6134-48d0-a722-fcd5656e0977 None None] MFA Rules not processed for user `725fae5c36ef41ab97e3d6c011400f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:52.309980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d0210d69-6134-48d0-a722-fcd5656e0977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:52.549533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1111/2238] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:52 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:52.622320 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23162d0-c2ee-4a23-94dc-3f77ee1f5b76 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:52.625858 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-d23162d0-c2ee-4a23-94dc-3f77ee1f5b76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:52.775915 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23162d0-c2ee-4a23-94dc-3f77ee1f5b76 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:52.781573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1859378-94c8-44b0-9703-e20f560f5845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:52.782463 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1859378-94c8-44b0-9703-e20f560f5845 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:52.782982 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1859378-94c8-44b0-9703-e20f560f5845 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:52.783365 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1859378-94c8-44b0-9703-e20f560f5845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:52.802624 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1859378-94c8-44b0-9703-e20f560f5845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:52.805289 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1859378-94c8-44b0-9703-e20f560f5845 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:52.838080 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1128/2239] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:52 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:15:52.922323 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0210d69-6134-48d0-a722-fcd5656e0977 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:52.924532 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d0210d69-6134-48d0-a722-fcd5656e0977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:52.955571 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0210d69-6134-48d0-a722-fcd5656e0977 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:52.963907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c91906-c326-4340-9895-cb76448caa39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:52.965130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c91906-c326-4340-9895-cb76448caa39 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:52.965750 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c91906-c326-4340-9895-cb76448caa39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:52.966193 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c91906-c326-4340-9895-cb76448caa39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:53.173599 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c91906-c326-4340-9895-cb76448caa39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:53.177301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c91906-c326-4340-9895-cb76448caa39 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:53.202662 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1112/2240] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:52 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:59.046049 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1859378-94c8-44b0-9703-e20f560f5845 service cinder] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:59.579857 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1859378-94c8-44b0-9703-e20f560f5845 service cinder] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:59.588707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7670f-b36e-48fd-83ac-4e36c9fd1515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:59.589900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7670f-b36e-48fd-83ac-4e36c9fd1515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:59.590350 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7670f-b36e-48fd-83ac-4e36c9fd1515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:59.590631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7670f-b36e-48fd-83ac-4e36c9fd1515 admin admin] PATH_INFO: `/v3/users/02973bd442144ceb9b6bed4acca45c73` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:59.592300 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7670f-b36e-48fd-83ac-4e36c9fd1515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02973bd442144ceb9b6bed4acca45c73)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:59.595976 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7670f-b36e-48fd-83ac-4e36c9fd1515 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:59.606528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:59.789718 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-16c7670f-b36e-48fd-83ac-4e36c9fd1515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02973bd442144ceb9b6bed4acca45c73'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:59.820710 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-16c7670f-b36e-48fd-83ac-4e36c9fd1515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02973bd442144ceb9b6bed4acca45c73'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:59.839517 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-16c7670f-b36e-48fd-83ac-4e36c9fd1515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02973bd442144ceb9b6bed4acca45c73'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:59.845629 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-16c7670f-b36e-48fd-83ac-4e36c9fd1515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02973bd442144ceb9b6bed4acca45c73'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:59.847960 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-16c7670f-b36e-48fd-83ac-4e36c9fd1515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02973bd442144ceb9b6bed4acca45c73'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:59.894441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1129/2241] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:15:59 2022] DELETE /identity/v3/users/02973bd442144ceb9b6bed4acca45c73 => generated 0 bytes in 846 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:59.896414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9b98d-1355-4b24-9136-e89d0e612ee8 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:59.897402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9b98d-1355-4b24-9136-e89d0e612ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:59.897842 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9b98d-1355-4b24-9136-e89d0e612ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:59.976663 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-a2d9b98d-1355-4b24-9136-e89d0e612ee8 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:59.979542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a2d9b98d-1355-4b24-9136-e89d0e612ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:00.218623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:00.252511 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1113/2242] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:16:00.275594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7670f-b36e-48fd-83ac-4e36c9fd1515 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:00.277708 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-16c7670f-b36e-48fd-83ac-4e36c9fd1515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:00.464593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7670f-b36e-48fd-83ac-4e36c9fd1515 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:00.478703 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cde42-d69a-4bf1-827e-3101408f06c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:00.479732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cde42-d69a-4bf1-827e-3101408f06c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:00.480706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cde42-d69a-4bf1-827e-3101408f06c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:00.481401 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cde42-d69a-4bf1-827e-3101408f06c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:00.494009 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cde42-d69a-4bf1-827e-3101408f06c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:00.495905 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cde42-d69a-4bf1-827e-3101408f06c0 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:00.509469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1130/2243] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:01.484842 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9b98d-1355-4b24-9136-e89d0e612ee8 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:01.487294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a2d9b98d-1355-4b24-9136-e89d0e612ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:01.537387 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9b98d-1355-4b24-9136-e89d0e612ee8 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:01.548027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f16e23-0fd0-42a2-a9b9-87c391636b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:01.549804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f16e23-0fd0-42a2-a9b9-87c391636b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:01.550454 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f16e23-0fd0-42a2-a9b9-87c391636b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:01.551034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f16e23-0fd0-42a2-a9b9-87c391636b05 admin admin] PATH_INFO: `/v3/projects/2b34e6c52c70497286e957db51e06b21` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:01.553914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f16e23-0fd0-42a2-a9b9-87c391636b05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b34e6c52c70497286e957db51e06b21)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:01.556561 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f16e23-0fd0-42a2-a9b9-87c391636b05 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:01.666961 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-83f16e23-0fd0-42a2-a9b9-87c391636b05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b34e6c52c70497286e957db51e06b21', 'request_id': 'req-83f16e23-0fd0-42a2-a9b9-87c391636b05'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:01.680635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-83f16e23-0fd0-42a2-a9b9-87c391636b05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b34e6c52c70497286e957db51e06b21', 'request_id': 'req-83f16e23-0fd0-42a2-a9b9-87c391636b05'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:01.698999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-83f16e23-0fd0-42a2-a9b9-87c391636b05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b34e6c52c70497286e957db51e06b21', 'request_id': 'req-83f16e23-0fd0-42a2-a9b9-87c391636b05'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:01.737634 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-83f16e23-0fd0-42a2-a9b9-87c391636b05 admin admin] The token cache is being invalidate because project 2b34e6c52c70497286e957db51e06b21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:01.740090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-83f16e23-0fd0-42a2-a9b9-87c391636b05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:01.743168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-83f16e23-0fd0-42a2-a9b9-87c391636b05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:01.753387 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1114/2244] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:16:01 2022] DELETE /identity/v3/projects/2b34e6c52c70497286e957db51e06b21 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:01.755524 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cde42-d69a-4bf1-827e-3101408f06c0 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:01.788927 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cde42-d69a-4bf1-827e-3101408f06c0 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:02.027702 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56017ea-5ea9-4fb8-a850-ea4b8a487f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:02.029444 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56017ea-5ea9-4fb8-a850-ea4b8a487f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:02.036187 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56017ea-5ea9-4fb8-a850-ea4b8a487f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:02.037034 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56017ea-5ea9-4fb8-a850-ea4b8a487f47 admin admin] PATH_INFO: `/v3/users/725fae5c36ef41ab97e3d6c011400f3a` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:02.045187 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56017ea-5ea9-4fb8-a850-ea4b8a487f47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=725fae5c36ef41ab97e3d6c011400f3a)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:02.062310 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56017ea-5ea9-4fb8-a850-ea4b8a487f47 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:02.166309 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:02.213153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e56017ea-5ea9-4fb8-a850-ea4b8a487f47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '725fae5c36ef41ab97e3d6c011400f3a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:02.233574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e56017ea-5ea9-4fb8-a850-ea4b8a487f47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '725fae5c36ef41ab97e3d6c011400f3a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:02.250265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e56017ea-5ea9-4fb8-a850-ea4b8a487f47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '725fae5c36ef41ab97e3d6c011400f3a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:02.252237 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e56017ea-5ea9-4fb8-a850-ea4b8a487f47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '725fae5c36ef41ab97e3d6c011400f3a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:02.253317 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e56017ea-5ea9-4fb8-a850-ea4b8a487f47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '725fae5c36ef41ab97e3d6c011400f3a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:02.283326 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1131/2245] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:16:01 2022] DELETE /identity/v3/users/725fae5c36ef41ab97e3d6c011400f3a => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:02.651649 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f16e23-0fd0-42a2-a9b9-87c391636b05 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:02.653434 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-83f16e23-0fd0-42a2-a9b9-87c391636b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:02.788411 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f16e23-0fd0-42a2-a9b9-87c391636b05 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:02.884717 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686a7b8-512b-4025-97ba-d0b31cdeb242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:02.885457 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686a7b8-512b-4025-97ba-d0b31cdeb242 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:02.885921 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686a7b8-512b-4025-97ba-d0b31cdeb242 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:02.886274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686a7b8-512b-4025-97ba-d0b31cdeb242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:02.888881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-3686a7b8-512b-4025-97ba-d0b31cdeb242 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:02.980408 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:03.007604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686a7b8-512b-4025-97ba-d0b31cdeb242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:03.011224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686a7b8-512b-4025-97ba-d0b31cdeb242 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:03.032632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1115/2246] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:04.223400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56017ea-5ea9-4fb8-a850-ea4b8a487f47 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:04.226884 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e56017ea-5ea9-4fb8-a850-ea4b8a487f47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:04.406108 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56017ea-5ea9-4fb8-a850-ea4b8a487f47 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:04.415592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9922b5e-12d9-4e4f-a587-7a73dfe4fffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:04.416991 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9922b5e-12d9-4e4f-a587-7a73dfe4fffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:04.417614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9922b5e-12d9-4e4f-a587-7a73dfe4fffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:04.418247 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9922b5e-12d9-4e4f-a587-7a73dfe4fffa admin admin] PATH_INFO: `/v3/projects/8ba09803de5b4e52b4dc2993da623f05` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:04.419917 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9922b5e-12d9-4e4f-a587-7a73dfe4fffa admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ba09803de5b4e52b4dc2993da623f05)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:04.422267 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9922b5e-12d9-4e4f-a587-7a73dfe4fffa admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:04.515427 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d9922b5e-12d9-4e4f-a587-7a73dfe4fffa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ba09803de5b4e52b4dc2993da623f05', 'request_id': 'req-d9922b5e-12d9-4e4f-a587-7a73dfe4fffa'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:04.528640 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d9922b5e-12d9-4e4f-a587-7a73dfe4fffa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ba09803de5b4e52b4dc2993da623f05', 'request_id': 'req-d9922b5e-12d9-4e4f-a587-7a73dfe4fffa'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:04.539056 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d9922b5e-12d9-4e4f-a587-7a73dfe4fffa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ba09803de5b4e52b4dc2993da623f05', 'request_id': 'req-d9922b5e-12d9-4e4f-a587-7a73dfe4fffa'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:04.555648 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d9922b5e-12d9-4e4f-a587-7a73dfe4fffa admin admin] The token cache is being invalidate because project 8ba09803de5b4e52b4dc2993da623f05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:04.555950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d9922b5e-12d9-4e4f-a587-7a73dfe4fffa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:04.556863 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d9922b5e-12d9-4e4f-a587-7a73dfe4fffa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:04.559063 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1132/2247] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:16:04 2022] DELETE /identity/v3/projects/8ba09803de5b4e52b4dc2993da623f05 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:04.567668 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0ec99-9089-40a2-b895-3ec75832a700 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:04.568300 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0ec99-9089-40a2-b895-3ec75832a700 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:04.569196 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0ec99-9089-40a2-b895-3ec75832a700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:04.618819 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-54a0ec99-9089-40a2-b895-3ec75832a700 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:04.621162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-54a0ec99-9089-40a2-b895-3ec75832a700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:04.768671 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1116/2248] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:04.782475 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3740b6-ccd5-4882-a9c3-7ae7f1d86c48 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:04.782792 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3740b6-ccd5-4882-a9c3-7ae7f1d86c48 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:04.783271 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3740b6-ccd5-4882-a9c3-7ae7f1d86c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:04.825432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-2c3740b6-ccd5-4882-a9c3-7ae7f1d86c48 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:04.827068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-2c3740b6-ccd5-4882-a9c3-7ae7f1d86c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:04.842675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1133/2249] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:04.847758 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0ec99-9089-40a2-b895-3ec75832a700 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:04.868631 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0ec99-9089-40a2-b895-3ec75832a700 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:04.878674 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8445d92-20c7-4900-83a2-bf4703ed9bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:04.879410 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8445d92-20c7-4900-83a2-bf4703ed9bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:04.879757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8445d92-20c7-4900-83a2-bf4703ed9bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:04.880268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8445d92-20c7-4900-83a2-bf4703ed9bc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:04.881100 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8445d92-20c7-4900-83a2-bf4703ed9bc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:16:04.881459 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8445d92-20c7-4900-83a2-bf4703ed9bc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:04.883282 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8445d92-20c7-4900-83a2-bf4703ed9bc7 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:04.901540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1117/2250] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:16:04.908028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3740b6-ccd5-4882-a9c3-7ae7f1d86c48 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:04.934750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3740b6-ccd5-4882-a9c3-7ae7f1d86c48 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:04.944251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09777fc8-688a-4e15-81c4-85ca8bb60114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:04.944802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09777fc8-688a-4e15-81c4-85ca8bb60114 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:04.945153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09777fc8-688a-4e15-81c4-85ca8bb60114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:04.945510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09777fc8-688a-4e15-81c4-85ca8bb60114 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:04.946331 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09777fc8-688a-4e15-81c4-85ca8bb60114 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:04.948570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09777fc8-688a-4e15-81c4-85ca8bb60114 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:05.000725 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1134/2251] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:16:04 2022] POST /identity/v3/projects => generated 369 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:05.005858 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8445d92-20c7-4900-83a2-bf4703ed9bc7 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:05.018912 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8445d92-20c7-4900-83a2-bf4703ed9bc7 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:05.099413 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8da564-ac54-4f20-8d6e-0992aebbc6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:05.100344 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8da564-ac54-4f20-8d6e-0992aebbc6db admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:05.100760 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8da564-ac54-4f20-8d6e-0992aebbc6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:05.101070 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8da564-ac54-4f20-8d6e-0992aebbc6db admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:05.101839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8da564-ac54-4f20-8d6e-0992aebbc6db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:05.103878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8da564-ac54-4f20-8d6e-0992aebbc6db admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:05.135583 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1118/2252] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:16:05 2022] POST /identity/v3/users => generated 285 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:16:05.143603 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09777fc8-688a-4e15-81c4-85ca8bb60114 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:05.164543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09777fc8-688a-4e15-81c4-85ca8bb60114 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:05.174171 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581fc33-8b4b-4cff-986f-f8916f2d832c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:05.174979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581fc33-8b4b-4cff-986f-f8916f2d832c admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:05.175480 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581fc33-8b4b-4cff-986f-f8916f2d832c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:05.175803 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581fc33-8b4b-4cff-986f-f8916f2d832c admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:05.177123 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581fc33-8b4b-4cff-986f-f8916f2d832c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:05.179485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581fc33-8b4b-4cff-986f-f8916f2d832c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:05.182395 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:05.199091 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1135/2253] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:05.205167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8da564-ac54-4f20-8d6e-0992aebbc6db admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:05.225592 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8da564-ac54-4f20-8d6e-0992aebbc6db admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:05.232264 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f324b68-d904-44f8-9823-fd9bf07a5766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:05.233270 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f324b68-d904-44f8-9823-fd9bf07a5766 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:05.233636 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f324b68-d904-44f8-9823-fd9bf07a5766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:05.233988 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f324b68-d904-44f8-9823-fd9bf07a5766 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:05.234903 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f324b68-d904-44f8-9823-fd9bf07a5766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:05.237288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f324b68-d904-44f8-9823-fd9bf07a5766 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:05.258605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1119/2254] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:05.259993 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581fc33-8b4b-4cff-986f-f8916f2d832c admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:05.274339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581fc33-8b4b-4cff-986f-f8916f2d832c admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:05.284676 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87eefb9-1883-4e74-9bb3-11a4c25d7c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:05.286388 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87eefb9-1883-4e74-9bb3-11a4c25d7c4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:05.286764 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87eefb9-1883-4e74-9bb3-11a4c25d7c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:05.287212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87eefb9-1883-4e74-9bb3-11a4c25d7c4e admin admin] PATH_INFO: `/v3/projects/7d21dda10b3e454d95490ac5b060d922/users/5b6973dc0c7e441c8374877ea80dcff1/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:05.319074 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87eefb9-1883-4e74-9bb3-11a4c25d7c4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d21dda10b3e454d95490ac5b060d922, user_id=5b6973dc0c7e441c8374877ea80dcff1, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:05.329488 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87eefb9-1883-4e74-9bb3-11a4c25d7c4e admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:05.378504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1136/2255] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:16:05 2022] PUT /identity/v3/projects/7d21dda10b3e454d95490ac5b060d922/users/5b6973dc0c7e441c8374877ea80dcff1/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:16:05.385116 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce745403-3978-443b-98fe-6e337ccd2b85 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:05.385896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce745403-3978-443b-98fe-6e337ccd2b85 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:05.386513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce745403-3978-443b-98fe-6e337ccd2b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:05.446457 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-ce745403-3978-443b-98fe-6e337ccd2b85 None None] MFA Rules not processed for user `5b6973dc0c7e441c8374877ea80dcff1`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:05.448074 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ce745403-3978-443b-98fe-6e337ccd2b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:05.517734 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:05.565721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1120/2256] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:05 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:05.585146 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87eefb9-1883-4e74-9bb3-11a4c25d7c4e admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:05.588064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f87eefb9-1883-4e74-9bb3-11a4c25d7c4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:05.598630 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce745403-3978-443b-98fe-6e337ccd2b85 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:05.601367 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ce745403-3978-443b-98fe-6e337ccd2b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:05.695781 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87eefb9-1883-4e74-9bb3-11a4c25d7c4e admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:05.709979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb51544-8072-4d66-8349-be18c2a2b90b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:05.711318 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb51544-8072-4d66-8349-be18c2a2b90b service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:05.711602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb51544-8072-4d66-8349-be18c2a2b90b service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:05.711959 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb51544-8072-4d66-8349-be18c2a2b90b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:05.726226 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb51544-8072-4d66-8349-be18c2a2b90b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:05.728565 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb51544-8072-4d66-8349-be18c2a2b90b service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:05.742470 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1137/2257] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:05 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:16:05.765860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce745403-3978-443b-98fe-6e337ccd2b85 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:05.787131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3566e1d-4c8d-402f-8df1-49ae2e6b60f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:05.788846 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3566e1d-4c8d-402f-8df1-49ae2e6b60f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:05.789815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3566e1d-4c8d-402f-8df1-49ae2e6b60f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:05.790168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb51544-8072-4d66-8349-be18c2a2b90b service cinder] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:05.790929 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3566e1d-4c8d-402f-8df1-49ae2e6b60f8 admin admin] PATH_INFO: `/v3/users/242563abc4214eb4a0f34ca4a320166c` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:05.794204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3566e1d-4c8d-402f-8df1-49ae2e6b60f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=242563abc4214eb4a0f34ca4a320166c)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:05.802432 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3566e1d-4c8d-402f-8df1-49ae2e6b60f8 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:05.816423 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb51544-8072-4d66-8349-be18c2a2b90b service cinder] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:05.825375 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f00c5-fea7-48d5-a73c-b04a71f96296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:05.827312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f00c5-fea7-48d5-a73c-b04a71f96296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:05.827580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f00c5-fea7-48d5-a73c-b04a71f96296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:05.828213 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f00c5-fea7-48d5-a73c-b04a71f96296 admin admin] PATH_INFO: `/v3/users/5b6973dc0c7e441c8374877ea80dcff1` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:05.831766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f00c5-fea7-48d5-a73c-b04a71f96296 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b6973dc0c7e441c8374877ea80dcff1)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:05.836160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f00c5-fea7-48d5-a73c-b04a71f96296 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:05.951604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a3566e1d-4c8d-402f-8df1-49ae2e6b60f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '242563abc4214eb4a0f34ca4a320166c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:05.954033 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a3566e1d-4c8d-402f-8df1-49ae2e6b60f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '242563abc4214eb4a0f34ca4a320166c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:05.979665 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a3566e1d-4c8d-402f-8df1-49ae2e6b60f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '242563abc4214eb4a0f34ca4a320166c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:05.981352 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a3566e1d-4c8d-402f-8df1-49ae2e6b60f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '242563abc4214eb4a0f34ca4a320166c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:05.997292 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-932f00c5-fea7-48d5-a73c-b04a71f96296 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b6973dc0c7e441c8374877ea80dcff1'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:06.022271 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-932f00c5-fea7-48d5-a73c-b04a71f96296 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b6973dc0c7e441c8374877ea80dcff1'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:06.030965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a3566e1d-4c8d-402f-8df1-49ae2e6b60f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '242563abc4214eb4a0f34ca4a320166c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:06.042541 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-932f00c5-fea7-48d5-a73c-b04a71f96296 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b6973dc0c7e441c8374877ea80dcff1'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:06.043589 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-932f00c5-fea7-48d5-a73c-b04a71f96296 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b6973dc0c7e441c8374877ea80dcff1'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:06.044890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-932f00c5-fea7-48d5-a73c-b04a71f96296 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b6973dc0c7e441c8374877ea80dcff1'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:06.071460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1121/2258] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:16:05 2022] DELETE /identity/v3/users/242563abc4214eb4a0f34ca4a320166c => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:06.076363 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96620035-44eb-4bd5-80f1-740b94c571bd None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:06.076686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96620035-44eb-4bd5-80f1-740b94c571bd None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:06.077030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96620035-44eb-4bd5-80f1-740b94c571bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:06.110677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-96620035-44eb-4bd5-80f1-740b94c571bd None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:06.114615 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-96620035-44eb-4bd5-80f1-740b94c571bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:06.131554 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1138/2259] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:16:05 2022] DELETE /identity/v3/users/5b6973dc0c7e441c8374877ea80dcff1 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:06.138196 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccc3cf-f634-41d4-8806-841fa49d3d22 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:06.138609 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccc3cf-f634-41d4-8806-841fa49d3d22 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:06.138997 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccc3cf-f634-41d4-8806-841fa49d3d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:06.199543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-6fccc3cf-f634-41d4-8806-841fa49d3d22 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:06.206903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-6fccc3cf-f634-41d4-8806-841fa49d3d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:06.231132 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1122/2260] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:16:06.293190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:06.346515 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1139/2261] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:06.380614 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96620035-44eb-4bd5-80f1-740b94c571bd None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:06.382192 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-96620035-44eb-4bd5-80f1-740b94c571bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:06.389160 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:06.480543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96620035-44eb-4bd5-80f1-740b94c571bd None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:06.597455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff41ba1-559b-4ced-ae71-b632729f30f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:06.599666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff41ba1-559b-4ced-ae71-b632729f30f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:06.600919 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff41ba1-559b-4ced-ae71-b632729f30f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:06.601500 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff41ba1-559b-4ced-ae71-b632729f30f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:06.618271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff41ba1-559b-4ced-ae71-b632729f30f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:06.620694 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff41ba1-559b-4ced-ae71-b632729f30f5 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:06.636288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1123/2262] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:06.747164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccc3cf-f634-41d4-8806-841fa49d3d22 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:06.749758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-6fccc3cf-f634-41d4-8806-841fa49d3d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:06.818854 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccc3cf-f634-41d4-8806-841fa49d3d22 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:06.827599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc87e4-e939-4126-8f3e-e16e2c354463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:06.829546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc87e4-e939-4126-8f3e-e16e2c354463 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:06.829947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc87e4-e939-4126-8f3e-e16e2c354463 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:06.830490 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc87e4-e939-4126-8f3e-e16e2c354463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:06.844151 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc87e4-e939-4126-8f3e-e16e2c354463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:06.846611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc87e4-e939-4126-8f3e-e16e2c354463 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:06.863039 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1140/2263] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:16:07.630531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff41ba1-559b-4ced-ae71-b632729f30f5 service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:07.632630 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-6ff41ba1-559b-4ced-ae71-b632729f30f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:07.663031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff41ba1-559b-4ced-ae71-b632729f30f5 service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:07.673947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5a36c-9a70-45d5-abb1-01fab6361b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:07.675606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5a36c-9a70-45d5-abb1-01fab6361b69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:07.675606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5a36c-9a70-45d5-abb1-01fab6361b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:07.704537 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5a36c-9a70-45d5-abb1-01fab6361b69 admin admin] PATH_INFO: `/v3/projects/7d21dda10b3e454d95490ac5b060d922` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:07.707378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae5a36c-9a70-45d5-abb1-01fab6361b69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d21dda10b3e454d95490ac5b060d922)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:07.714008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae5a36c-9a70-45d5-abb1-01fab6361b69 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:07.872920 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0ae5a36c-9a70-45d5-abb1-01fab6361b69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d21dda10b3e454d95490ac5b060d922', 'request_id': 'req-0ae5a36c-9a70-45d5-abb1-01fab6361b69'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:07.887050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0ae5a36c-9a70-45d5-abb1-01fab6361b69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d21dda10b3e454d95490ac5b060d922', 'request_id': 'req-0ae5a36c-9a70-45d5-abb1-01fab6361b69'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:07.903083 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0ae5a36c-9a70-45d5-abb1-01fab6361b69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d21dda10b3e454d95490ac5b060d922', 'request_id': 'req-0ae5a36c-9a70-45d5-abb1-01fab6361b69'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:07.932424 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0ae5a36c-9a70-45d5-abb1-01fab6361b69 admin admin] The token cache is being invalidate because project 7d21dda10b3e454d95490ac5b060d922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:07.933126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0ae5a36c-9a70-45d5-abb1-01fab6361b69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:07.934371 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0ae5a36c-9a70-45d5-abb1-01fab6361b69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:07.937298 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1124/2264] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:16:07 2022] DELETE /identity/v3/projects/7d21dda10b3e454d95490ac5b060d922 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:07.944856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33287ca6-486d-4219-a457-594e24157960 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:07.945739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33287ca6-486d-4219-a457-594e24157960 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:07.945739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33287ca6-486d-4219-a457-594e24157960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:07.991404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-33287ca6-486d-4219-a457-594e24157960 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:07.992967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-33287ca6-486d-4219-a457-594e24157960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:08.163718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5a36c-9a70-45d5-abb1-01fab6361b69 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:08.166470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-0ae5a36c-9a70-45d5-abb1-01fab6361b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:08.219984 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1141/2265] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:08.236847 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a28e1-df3f-4888-8ae8-ae7472ec7040 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:08.237408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a28e1-df3f-4888-8ae8-ae7472ec7040 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:08.237801 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a28e1-df3f-4888-8ae8-ae7472ec7040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:08.316802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-120a28e1-df3f-4888-8ae8-ae7472ec7040 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:08.316802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-120a28e1-df3f-4888-8ae8-ae7472ec7040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:08.338306 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1142/2266] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:08.345505 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a28e1-df3f-4888-8ae8-ae7472ec7040 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:08.374713 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a28e1-df3f-4888-8ae8-ae7472ec7040 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:08.383797 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:08.386268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a968ab3-460d-4cc6-bb38-42b1e3a601d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:08.388894 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a968ab3-460d-4cc6-bb38-42b1e3a601d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:08.389481 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a968ab3-460d-4cc6-bb38-42b1e3a601d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:08.389979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a968ab3-460d-4cc6-bb38-42b1e3a601d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:08.391988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a968ab3-460d-4cc6-bb38-42b1e3a601d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:16:08.393412 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a968ab3-460d-4cc6-bb38-42b1e3a601d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:08.400198 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a968ab3-460d-4cc6-bb38-42b1e3a601d2 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:08.441445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1143/2267] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:16:08.447949 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a968ab3-460d-4cc6-bb38-42b1e3a601d2 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:08.470095 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5a36c-9a70-45d5-abb1-01fab6361b69 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:08.479906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a968ab3-460d-4cc6-bb38-42b1e3a601d2 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:08.480247 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c922acc-1968-44a7-abc6-8f182fe4cf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:08.481356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c922acc-1968-44a7-abc6-8f182fe4cf03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:08.482088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c922acc-1968-44a7-abc6-8f182fe4cf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:08.482583 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c922acc-1968-44a7-abc6-8f182fe4cf03 admin admin] PATH_INFO: `/v3/projects/7960c549fa3d43bcb7b29e9046767b2a` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:08.485067 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c922acc-1968-44a7-abc6-8f182fe4cf03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7960c549fa3d43bcb7b29e9046767b2a)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:08.489334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c922acc-1968-44a7-abc6-8f182fe4cf03 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:08.510284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651da47-be27-4b72-b4f5-c273fcb551a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:08.510284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651da47-be27-4b72-b4f5-c273fcb551a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:08.510284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651da47-be27-4b72-b4f5-c273fcb551a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:08.510284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651da47-be27-4b72-b4f5-c273fcb551a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:08.511021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651da47-be27-4b72-b4f5-c273fcb551a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:08.518345 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651da47-be27-4b72-b4f5-c273fcb551a6 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:08.619757 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1144/2268] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:16:08 2022] POST /identity/v3/projects => generated 359 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:08.623277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651da47-be27-4b72-b4f5-c273fcb551a6 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:08.636895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3c922acc-1968-44a7-abc6-8f182fe4cf03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7960c549fa3d43bcb7b29e9046767b2a', 'request_id': 'req-3c922acc-1968-44a7-abc6-8f182fe4cf03'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:08.664603 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3c922acc-1968-44a7-abc6-8f182fe4cf03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7960c549fa3d43bcb7b29e9046767b2a', 'request_id': 'req-3c922acc-1968-44a7-abc6-8f182fe4cf03'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:08.687312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3c922acc-1968-44a7-abc6-8f182fe4cf03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7960c549fa3d43bcb7b29e9046767b2a', 'request_id': 'req-3c922acc-1968-44a7-abc6-8f182fe4cf03'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:08.725314 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3c922acc-1968-44a7-abc6-8f182fe4cf03 admin admin] The token cache is being invalidate because project 7960c549fa3d43bcb7b29e9046767b2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:08.725994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3c922acc-1968-44a7-abc6-8f182fe4cf03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:08.740606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3c922acc-1968-44a7-abc6-8f182fe4cf03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:08.748090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1125/2269] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:16:08 2022] DELETE /identity/v3/projects/7960c549fa3d43bcb7b29e9046767b2a => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:16:08.753821 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef46b0b-8c8b-4d17-bc53-33baeeeaf1cc None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:08.754346 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef46b0b-8c8b-4d17-bc53-33baeeeaf1cc None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:08.756546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef46b0b-8c8b-4d17-bc53-33baeeeaf1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:08.834962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651da47-be27-4b72-b4f5-c273fcb551a6 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:08.847532 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc2093-b661-42fc-a219-498f1f7f6414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:08.848833 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc2093-b661-42fc-a219-498f1f7f6414 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:08.849784 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc2093-b661-42fc-a219-498f1f7f6414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:08.850324 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc2093-b661-42fc-a219-498f1f7f6414 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:08.851907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc2093-b661-42fc-a219-498f1f7f6414 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:08.855949 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc2093-b661-42fc-a219-498f1f7f6414 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:08.865330 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-7ef46b0b-8c8b-4d17-bc53-33baeeeaf1cc None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:08.870763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-7ef46b0b-8c8b-4d17-bc53-33baeeeaf1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:08.909560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1126/2270] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:16:08.912016 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6b5d7-2182-4b70-b3fe-d6265b51e811 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:08.913764 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6b5d7-2182-4b70-b3fe-d6265b51e811 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:08.913764 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6b5d7-2182-4b70-b3fe-d6265b51e811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:08.914810 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1145/2271] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:16:08 2022] POST /identity/v3/users => generated 280 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:08.921299 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc2093-b661-42fc-a219-498f1f7f6414 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:08.924245 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e1bc2093-b661-42fc-a219-498f1f7f6414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:08.948779 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc2093-b661-42fc-a219-498f1f7f6414 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:08.949814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-98c6b5d7-2182-4b70-b3fe-d6265b51e811 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:08.951697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-98c6b5d7-2182-4b70-b3fe-d6265b51e811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:08.959078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f3e67-0ed9-46b1-befa-74fa3827159c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:08.961398 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f3e67-0ed9-46b1-befa-74fa3827159c admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:08.962198 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f3e67-0ed9-46b1-befa-74fa3827159c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:08.962747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f3e67-0ed9-46b1-befa-74fa3827159c admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:08.964747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f3e67-0ed9-46b1-befa-74fa3827159c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:08.966848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1127/2272] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:08.969342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f3e67-0ed9-46b1-befa-74fa3827159c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:08.975060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6b5d7-2182-4b70-b3fe-d6265b51e811 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:09.013679 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1146/2273] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:09.018851 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f3e67-0ed9-46b1-befa-74fa3827159c admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:09.028510 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6b5d7-2182-4b70-b3fe-d6265b51e811 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:09.042533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb6cc5-7ef3-46bd-b0b1-dc7b56128b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:09.043926 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb6cc5-7ef3-46bd-b0b1-dc7b56128b33 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:09.044615 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb6cc5-7ef3-46bd-b0b1-dc7b56128b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:09.045316 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb6cc5-7ef3-46bd-b0b1-dc7b56128b33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:09.045859 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f3e67-0ed9-46b1-befa-74fa3827159c admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:09.048427 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb6cc5-7ef3-46bd-b0b1-dc7b56128b33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:16:09.049835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb6cc5-7ef3-46bd-b0b1-dc7b56128b33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:09.052847 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772a5e0-148f-4954-ad14-c15b6b7ffe33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:09.053833 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772a5e0-148f-4954-ad14-c15b6b7ffe33 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:09.054052 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb6cc5-7ef3-46bd-b0b1-dc7b56128b33 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:09.054244 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772a5e0-148f-4954-ad14-c15b6b7ffe33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:09.054542 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772a5e0-148f-4954-ad14-c15b6b7ffe33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:09.055886 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2772a5e0-148f-4954-ad14-c15b6b7ffe33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:09.057945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2772a5e0-148f-4954-ad14-c15b6b7ffe33 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:09.068794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:09.083846 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1147/2274] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:09.088653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772a5e0-148f-4954-ad14-c15b6b7ffe33 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:09.105252 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1128/2275] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:16:09.112759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb6cc5-7ef3-46bd-b0b1-dc7b56128b33 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:09.124924 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772a5e0-148f-4954-ad14-c15b6b7ffe33 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:09.139944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7018ce7-9a45-4ecc-8627-6b5c72d8e5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:09.140861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7018ce7-9a45-4ecc-8627-6b5c72d8e5ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:09.141388 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7018ce7-9a45-4ecc-8627-6b5c72d8e5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:09.141777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7018ce7-9a45-4ecc-8627-6b5c72d8e5ac admin admin] PATH_INFO: `/v3/projects/e4cccf87feaa4da6931eed33256bf67c/users/608ccf5ded7d4b9482f7a59c785f8bd4/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:09.159163 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb6cc5-7ef3-46bd-b0b1-dc7b56128b33 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:09.175956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e93a4-2090-4315-9817-32a5de7feb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:09.177700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e93a4-2090-4315-9817-32a5de7feb8b admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:09.178006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e93a4-2090-4315-9817-32a5de7feb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:09.178279 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e93a4-2090-4315-9817-32a5de7feb8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:09.180330 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e93a4-2090-4315-9817-32a5de7feb8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:09.181171 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7018ce7-9a45-4ecc-8627-6b5c72d8e5ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4cccf87feaa4da6931eed33256bf67c, user_id=608ccf5ded7d4b9482f7a59c785f8bd4, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:09.186352 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e93a4-2090-4315-9817-32a5de7feb8b admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:09.196281 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7018ce7-9a45-4ecc-8627-6b5c72d8e5ac admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:09.228183 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1148/2276] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:16:09 2022] PUT /identity/v3/projects/e4cccf87feaa4da6931eed33256bf67c/users/608ccf5ded7d4b9482f7a59c785f8bd4/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:16:09.257342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66f8f8-5259-4f30-a0e2-41bc45b8ce04 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:09.257761 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66f8f8-5259-4f30-a0e2-41bc45b8ce04 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:09.258184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66f8f8-5259-4f30-a0e2-41bc45b8ce04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:09.333507 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-1e66f8f8-5259-4f30-a0e2-41bc45b8ce04 None None] MFA Rules not processed for user `608ccf5ded7d4b9482f7a59c785f8bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:09.334918 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-1e66f8f8-5259-4f30-a0e2-41bc45b8ce04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:09.363577 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1129/2277] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:16:09 2022] POST /identity/v3/projects => generated 371 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:09.369659 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e93a4-2090-4315-9817-32a5de7feb8b admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:09.509215 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1149/2278] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:09 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:16:09.593343 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66f8f8-5259-4f30-a0e2-41bc45b8ce04 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:09.595099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-1e66f8f8-5259-4f30-a0e2-41bc45b8ce04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:09.619551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e93a4-2090-4315-9817-32a5de7feb8b admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:09.627877 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f6db2-0f08-41c5-8b6d-2961fb9cdc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:09.629032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f6db2-0f08-41c5-8b6d-2961fb9cdc02 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:09.629606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f6db2-0f08-41c5-8b6d-2961fb9cdc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:09.630118 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f6db2-0f08-41c5-8b6d-2961fb9cdc02 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:09.631059 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f6db2-0f08-41c5-8b6d-2961fb9cdc02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:09.633814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f6db2-0f08-41c5-8b6d-2961fb9cdc02 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:09.690083 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1130/2279] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:16:09 2022] POST /identity/v3/users => generated 286 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:09.692910 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f6db2-0f08-41c5-8b6d-2961fb9cdc02 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:09.725914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f6db2-0f08-41c5-8b6d-2961fb9cdc02 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:09.745901 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56133666-d2df-4b76-ab48-69ca48e7d303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:09.746262 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56133666-d2df-4b76-ab48-69ca48e7d303 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:09.746575 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56133666-d2df-4b76-ab48-69ca48e7d303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:09.747073 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56133666-d2df-4b76-ab48-69ca48e7d303 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:09.748506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56133666-d2df-4b76-ab48-69ca48e7d303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:09.753095 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56133666-d2df-4b76-ab48-69ca48e7d303 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:09.772624 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66f8f8-5259-4f30-a0e2-41bc45b8ce04 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:09.784533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1131/2280] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:16:09.784957 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c60c9-e713-4735-af15-7ed6da21b079 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:09.787142 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c60c9-e713-4735-af15-7ed6da21b079 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:09.787700 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c60c9-e713-4735-af15-7ed6da21b079 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:09.788035 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c60c9-e713-4735-af15-7ed6da21b079 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:09.788161 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56133666-d2df-4b76-ab48-69ca48e7d303 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:09.810476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c60c9-e713-4735-af15-7ed6da21b079 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:09.813912 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c60c9-e713-4735-af15-7ed6da21b079 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:09.820688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:09.847688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56133666-d2df-4b76-ab48-69ca48e7d303 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:09.850848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1150/2281] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:09 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:16:09.904086 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d27232-ba8e-4904-ab39-d327e17642d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:09.906046 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d27232-ba8e-4904-ab39-d327e17642d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:09.909204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d27232-ba8e-4904-ab39-d327e17642d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:09.909204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d27232-ba8e-4904-ab39-d327e17642d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:09.909204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d27232-ba8e-4904-ab39-d327e17642d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:09.913405 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d27232-ba8e-4904-ab39-d327e17642d4 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:09.915356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c60c9-e713-4735-af15-7ed6da21b079 service cinder] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:09.955745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1132/2282] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:09.957546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c60c9-e713-4735-af15-7ed6da21b079 service cinder] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:09.959416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d27232-ba8e-4904-ab39-d327e17642d4 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:09.976974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360cbb2-0070-4a92-9506-91885c25540d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:09.978046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360cbb2-0070-4a92-9506-91885c25540d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:09.978501 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360cbb2-0070-4a92-9506-91885c25540d admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:09.979026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360cbb2-0070-4a92-9506-91885c25540d admin admin] PATH_INFO: `/v3/users/608ccf5ded7d4b9482f7a59c785f8bd4` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:09.983507 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d27232-ba8e-4904-ab39-d327e17642d4 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:09.988651 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360cbb2-0070-4a92-9506-91885c25540d admin admin] RBAC: Authorizing `identity:delete_user(user_id=608ccf5ded7d4b9482f7a59c785f8bd4)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:09.988651 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360cbb2-0070-4a92-9506-91885c25540d admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:10.000130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb8597-c833-4204-ba13-9f897d963a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:10.001722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb8597-c833-4204-ba13-9f897d963a4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:10.002284 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb8597-c833-4204-ba13-9f897d963a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:10.004412 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb8597-c833-4204-ba13-9f897d963a4c admin admin] PATH_INFO: `/v3/projects/8d1b2f228db54737a4a7901d6f2ab7c9/users/70353c6f18f4457096a0def6c0454c6e/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:10.043675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb8597-c833-4204-ba13-9f897d963a4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d1b2f228db54737a4a7901d6f2ab7c9, user_id=70353c6f18f4457096a0def6c0454c6e, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:10.048664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb8597-c833-4204-ba13-9f897d963a4c admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:10.097680 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1133/2283] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:16:09 2022] PUT /identity/v3/projects/8d1b2f228db54737a4a7901d6f2ab7c9/users/70353c6f18f4457096a0def6c0454c6e/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:16:10.118800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce810192-6dae-45f9-9e3e-7925d76b97f3 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:10.119292 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce810192-6dae-45f9-9e3e-7925d76b97f3 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:10.119788 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce810192-6dae-45f9-9e3e-7925d76b97f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:10.130312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a360cbb2-0070-4a92-9506-91885c25540d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '608ccf5ded7d4b9482f7a59c785f8bd4'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:10.146079 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a360cbb2-0070-4a92-9506-91885c25540d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '608ccf5ded7d4b9482f7a59c785f8bd4'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:10.165397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a360cbb2-0070-4a92-9506-91885c25540d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '608ccf5ded7d4b9482f7a59c785f8bd4'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:10.168065 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a360cbb2-0070-4a92-9506-91885c25540d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '608ccf5ded7d4b9482f7a59c785f8bd4'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:10.169395 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a360cbb2-0070-4a92-9506-91885c25540d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '608ccf5ded7d4b9482f7a59c785f8bd4'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:10.172802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:10.207053 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-ce810192-6dae-45f9-9e3e-7925d76b97f3 None None] MFA Rules not processed for user `70353c6f18f4457096a0def6c0454c6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:10.209886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ce810192-6dae-45f9-9e3e-7925d76b97f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:10.229129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1151/2284] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:16:09 2022] DELETE /identity/v3/users/608ccf5ded7d4b9482f7a59c785f8bd4 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:16:10.234021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd0af6-6e9c-4215-bae8-40f0d18fa0a9 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:10.234862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd0af6-6e9c-4215-bae8-40f0d18fa0a9 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:10.235576 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd0af6-6e9c-4215-bae8-40f0d18fa0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:10.278043 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-a1dd0af6-6e9c-4215-bae8-40f0d18fa0a9 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:10.280771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a1dd0af6-6e9c-4215-bae8-40f0d18fa0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:10.429802 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1134/2285] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:10.458613 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce810192-6dae-45f9-9e3e-7925d76b97f3 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:10.463402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ce810192-6dae-45f9-9e3e-7925d76b97f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:10.493261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1152/2286] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:10.511604 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd0af6-6e9c-4215-bae8-40f0d18fa0a9 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:10.513729 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a1dd0af6-6e9c-4215-bae8-40f0d18fa0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:10.666444 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd0af6-6e9c-4215-bae8-40f0d18fa0a9 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:10.680347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd0170-9bd7-4ad6-a652-d36bf6e7f6c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:10.681432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd0170-9bd7-4ad6-a652-d36bf6e7f6c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:10.681881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd0170-9bd7-4ad6-a652-d36bf6e7f6c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:10.682340 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd0170-9bd7-4ad6-a652-d36bf6e7f6c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:10.730862 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce810192-6dae-45f9-9e3e-7925d76b97f3 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:10.731906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd0170-9bd7-4ad6-a652-d36bf6e7f6c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:10.735341 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd0170-9bd7-4ad6-a652-d36bf6e7f6c1 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:10.788007 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1153/2287] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:10.932339 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76a62c-9ca5-48b7-8747-c4c91c7cbcbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:10.933724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76a62c-9ca5-48b7-8747-c4c91c7cbcbd service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:10.934334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76a62c-9ca5-48b7-8747-c4c91c7cbcbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:10.934730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76a62c-9ca5-48b7-8747-c4c91c7cbcbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:11.036421 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:11.151070 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76a62c-9ca5-48b7-8747-c4c91c7cbcbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:11.154032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76a62c-9ca5-48b7-8747-c4c91c7cbcbd service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:11.191396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1135/2288] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 735 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:16:11.274699 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd0170-9bd7-4ad6-a652-d36bf6e7f6c1 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:11.277548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-17bd0170-9bd7-4ad6-a652-d36bf6e7f6c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:11.317788 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd0170-9bd7-4ad6-a652-d36bf6e7f6c1 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:11.333525 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5718c21-eacc-4873-9f24-ce970058d84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:11.335956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5718c21-eacc-4873-9f24-ce970058d84a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:11.336943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5718c21-eacc-4873-9f24-ce970058d84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:11.337645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5718c21-eacc-4873-9f24-ce970058d84a admin admin] PATH_INFO: `/v3/users/70353c6f18f4457096a0def6c0454c6e` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:11.340778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5718c21-eacc-4873-9f24-ce970058d84a admin admin] RBAC: Authorizing `identity:delete_user(user_id=70353c6f18f4457096a0def6c0454c6e)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:11.345049 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5718c21-eacc-4873-9f24-ce970058d84a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:11.477654 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a5718c21-eacc-4873-9f24-ce970058d84a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70353c6f18f4457096a0def6c0454c6e'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:11.491118 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a5718c21-eacc-4873-9f24-ce970058d84a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70353c6f18f4457096a0def6c0454c6e'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:11.507344 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a5718c21-eacc-4873-9f24-ce970058d84a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70353c6f18f4457096a0def6c0454c6e'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:11.508014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a5718c21-eacc-4873-9f24-ce970058d84a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70353c6f18f4457096a0def6c0454c6e'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:11.508740 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a5718c21-eacc-4873-9f24-ce970058d84a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70353c6f18f4457096a0def6c0454c6e'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:11.555242 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:11.566513 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1154/2289] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:16:11 2022] DELETE /identity/v3/users/70353c6f18f4457096a0def6c0454c6e => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:11.574610 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de90cb8-7164-48c7-b491-b2571deac107 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:11.575135 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de90cb8-7164-48c7-b491-b2571deac107 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:11.576611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de90cb8-7164-48c7-b491-b2571deac107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:11.618909 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-8de90cb8-7164-48c7-b491-b2571deac107 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:11.621034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-8de90cb8-7164-48c7-b491-b2571deac107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:11.785006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1136/2290] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:11.804335 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5718c21-eacc-4873-9f24-ce970058d84a admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:11.806842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a5718c21-eacc-4873-9f24-ce970058d84a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:11.908975 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de90cb8-7164-48c7-b491-b2571deac107 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:11.912079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-8de90cb8-7164-48c7-b491-b2571deac107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:11.939094 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de90cb8-7164-48c7-b491-b2571deac107 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:11.949266 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a13a6-136a-4714-aea6-5492307c94a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:11.950650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a13a6-136a-4714-aea6-5492307c94a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:11.951261 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a13a6-136a-4714-aea6-5492307c94a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:11.951857 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a13a6-136a-4714-aea6-5492307c94a0 admin admin] PATH_INFO: `/v3/projects/e4cccf87feaa4da6931eed33256bf67c` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:11.956146 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a13a6-136a-4714-aea6-5492307c94a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4cccf87feaa4da6931eed33256bf67c)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:11.961532 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a13a6-136a-4714-aea6-5492307c94a0 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:12.041300 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5718c21-eacc-4873-9f24-ce970058d84a admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:12.050516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d434bdf-5b59-4aea-9abe-76980cb3c0e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:12.052037 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d434bdf-5b59-4aea-9abe-76980cb3c0e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:12.052947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d434bdf-5b59-4aea-9abe-76980cb3c0e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:12.053883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d434bdf-5b59-4aea-9abe-76980cb3c0e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:12.079648 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d434bdf-5b59-4aea-9abe-76980cb3c0e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:12.083389 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d434bdf-5b59-4aea-9abe-76980cb3c0e7 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:12.106069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a99a13a6-136a-4714-aea6-5492307c94a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4cccf87feaa4da6931eed33256bf67c', 'request_id': 'req-a99a13a6-136a-4714-aea6-5492307c94a0'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:12.112529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1155/2291] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:12.124866 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a99a13a6-136a-4714-aea6-5492307c94a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4cccf87feaa4da6931eed33256bf67c', 'request_id': 'req-a99a13a6-136a-4714-aea6-5492307c94a0'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:12.138921 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a99a13a6-136a-4714-aea6-5492307c94a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4cccf87feaa4da6931eed33256bf67c', 'request_id': 'req-a99a13a6-136a-4714-aea6-5492307c94a0'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:12.169539 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a99a13a6-136a-4714-aea6-5492307c94a0 admin admin] The token cache is being invalidate because project e4cccf87feaa4da6931eed33256bf67c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:12.170193 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a99a13a6-136a-4714-aea6-5492307c94a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:12.172126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a99a13a6-136a-4714-aea6-5492307c94a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:12.181821 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1137/2292] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:16:11 2022] DELETE /identity/v3/projects/e4cccf87feaa4da6931eed33256bf67c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:12.184423 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd29add-f0ef-4d3b-88ca-367d8a120165 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:12.184900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd29add-f0ef-4d3b-88ca-367d8a120165 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:12.185357 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd29add-f0ef-4d3b-88ca-367d8a120165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:12.234957 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-ebd29add-f0ef-4d3b-88ca-367d8a120165 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:12.241657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ebd29add-f0ef-4d3b-88ca-367d8a120165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:12.297284 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771273a-5ee9-47f0-b7a1-27fc10715041 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:12.297937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771273a-5ee9-47f0-b7a1-27fc10715041 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:12.298573 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771273a-5ee9-47f0-b7a1-27fc10715041 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:12.301569 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1138/2293] 10.222.0.93 () {54 vars in 921 bytes} [Mon Aug 1 15:16:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:12.314458 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282e5c2-b5f9-4ed7-b0a1-c364dcf00f27 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:12.315105 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282e5c2-b5f9-4ed7-b0a1-c364dcf00f27 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:12.315819 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282e5c2-b5f9-4ed7-b0a1-c364dcf00f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:12.384904 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-1282e5c2-b5f9-4ed7-b0a1-c364dcf00f27 None None] MFA Rules not processed for user `3d9be74621d14c94b95ae94947e5abbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:12.387651 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-1282e5c2-b5f9-4ed7-b0a1-c364dcf00f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:12.453717 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1156/2294] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:16:12.476048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edb5cf-d345-4c36-aac2-5fe07f13c7ed None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:12.477271 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edb5cf-d345-4c36-aac2-5fe07f13c7ed None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:12.477994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edb5cf-d345-4c36-aac2-5fe07f13c7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:12.546682 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-50edb5cf-d345-4c36-aac2-5fe07f13c7ed None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:12.549312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-50edb5cf-d345-4c36-aac2-5fe07f13c7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:12.571171 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1157/2295] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:12.578067 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edb5cf-d345-4c36-aac2-5fe07f13c7ed None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:12.593128 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:12.619106 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edb5cf-d345-4c36-aac2-5fe07f13c7ed None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:12.637457 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4bc3b5-f7d2-4085-af08-bf509d38da3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:12.638748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bc3b5-f7d2-4085-af08-bf509d38da3c admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:12.640073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bc3b5-f7d2-4085-af08-bf509d38da3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:12.641068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bc3b5-f7d2-4085-af08-bf509d38da3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:12.642324 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4bc3b5-f7d2-4085-af08-bf509d38da3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:16:12.643001 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4bc3b5-f7d2-4085-af08-bf509d38da3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:12.647047 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4bc3b5-f7d2-4085-af08-bf509d38da3c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:12.677310 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1139/2296] 10.222.0.93 () {58 vars in 1018 bytes} [Mon Aug 1 15:16:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:16:12.691815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1158/2297] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:16:12.699719 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282e5c2-b5f9-4ed7-b0a1-c364dcf00f27 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:12.703654 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4bc3b5-f7d2-4085-af08-bf509d38da3c admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:12.707913 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3c4bc3b5-f7d2-4085-af08-bf509d38da3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:12.726794 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282e5c2-b5f9-4ed7-b0a1-c364dcf00f27 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:12.738355 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613834e3-2804-4924-81ea-206ed8993463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:12.741257 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613834e3-2804-4924-81ea-206ed8993463 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:12.741931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613834e3-2804-4924-81ea-206ed8993463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:12.742499 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613834e3-2804-4924-81ea-206ed8993463 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:12.744326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613834e3-2804-4924-81ea-206ed8993463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:12.750331 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613834e3-2804-4924-81ea-206ed8993463 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:12.838843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1140/2298] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:16:12 2022] POST /identity/v3/projects => generated 377 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:12.849250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613834e3-2804-4924-81ea-206ed8993463 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:12.935312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4bc3b5-f7d2-4085-af08-bf509d38da3c admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:13.059931 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:13.071600 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613834e3-2804-4924-81ea-206ed8993463 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:13.086564 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52912ee6-d51b-44c7-b349-1226399061f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:13.091015 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52912ee6-d51b-44c7-b349-1226399061f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:13.091503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52912ee6-d51b-44c7-b349-1226399061f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:13.091977 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52912ee6-d51b-44c7-b349-1226399061f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:13.093602 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52912ee6-d51b-44c7-b349-1226399061f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:13.097679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52912ee6-d51b-44c7-b349-1226399061f2 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:13.156541 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1141/2299] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:16:12 2022] POST /identity/v3/users => generated 289 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:13.163333 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52912ee6-d51b-44c7-b349-1226399061f2 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:13.165479 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb0fb8-49f1-40a3-b690-0e8c819970c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4461a389da5848f5afd9854758707829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:13.167171 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb0fb8-49f1-40a3-b690-0e8c819970c7 service glance] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:13.167171 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb0fb8-49f1-40a3-b690-0e8c819970c7 service glance] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:13.167562 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb0fb8-49f1-40a3-b690-0e8c819970c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:13.231407 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52912ee6-d51b-44c7-b349-1226399061f2 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:13.273880 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab5b8b-4a41-44d2-8eff-fbf23988cac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:13.274706 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab5b8b-4a41-44d2-8eff-fbf23988cac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:13.275143 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab5b8b-4a41-44d2-8eff-fbf23988cac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:13.275776 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab5b8b-4a41-44d2-8eff-fbf23988cac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:13.276992 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab5b8b-4a41-44d2-8eff-fbf23988cac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:13.282407 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab5b8b-4a41-44d2-8eff-fbf23988cac1 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:13.323652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1142/2300] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:13.325704 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab5b8b-4a41-44d2-8eff-fbf23988cac1 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:13.369756 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab5b8b-4a41-44d2-8eff-fbf23988cac1 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:13.380797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb0fb8-49f1-40a3-b690-0e8c819970c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:13.384510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb0fb8-49f1-40a3-b690-0e8c819970c7 service glance] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:13.400704 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb623f7-f9fc-4357-b821-84075888fa50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:13.402014 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb623f7-f9fc-4357-b821-84075888fa50 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:13.402707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb623f7-f9fc-4357-b821-84075888fa50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:13.403346 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb623f7-f9fc-4357-b821-84075888fa50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:13.404817 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb623f7-f9fc-4357-b821-84075888fa50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:13.407174 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb623f7-f9fc-4357-b821-84075888fa50 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:13.413134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1159/2301] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 710 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:13.431409 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1143/2302] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:13.436791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb0fb8-49f1-40a3-b690-0e8c819970c7 service glance] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:13.483684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb0fb8-49f1-40a3-b690-0e8c819970c7 service glance] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:13.553451 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37efe53-9e88-4249-bd90-9987aa2bafe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:13.554271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb623f7-f9fc-4357-b821-84075888fa50 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:13.554452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37efe53-9e88-4249-bd90-9987aa2bafe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:13.554840 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37efe53-9e88-4249-bd90-9987aa2bafe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:13.555382 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37efe53-9e88-4249-bd90-9987aa2bafe2 admin admin] PATH_INFO: `/v3/projects/2842c4382a22430497f28e814c59560a/users/cd68383bd497477bb2d2292e814d799a/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:13.557088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-cbb623f7-f9fc-4357-b821-84075888fa50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:13.620751 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb623f7-f9fc-4357-b821-84075888fa50 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:13.628198 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37efe53-9e88-4249-bd90-9987aa2bafe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2842c4382a22430497f28e814c59560a, user_id=cd68383bd497477bb2d2292e814d799a, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:13.639358 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37efe53-9e88-4249-bd90-9987aa2bafe2 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:13.659291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833b25f-47f0-4fe9-8ad4-43882dcfe536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:13.663515 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833b25f-47f0-4fe9-8ad4-43882dcfe536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:13.664721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833b25f-47f0-4fe9-8ad4-43882dcfe536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:13.665528 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833b25f-47f0-4fe9-8ad4-43882dcfe536 admin admin] PATH_INFO: `/v3/projects/8d1b2f228db54737a4a7901d6f2ab7c9` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:13.668263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833b25f-47f0-4fe9-8ad4-43882dcfe536 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d1b2f228db54737a4a7901d6f2ab7c9)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:13.673955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833b25f-47f0-4fe9-8ad4-43882dcfe536 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:13.692948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1160/2303] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:16:13 2022] PUT /identity/v3/projects/2842c4382a22430497f28e814c59560a/users/cd68383bd497477bb2d2292e814d799a/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:13.696189 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605a8d0-aa3e-4456-b16b-02d5ffa79c89 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:13.698257 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605a8d0-aa3e-4456-b16b-02d5ffa79c89 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:13.699011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605a8d0-aa3e-4456-b16b-02d5ffa79c89 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:13.702132 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1161/2304] 10.222.0.93 () {54 vars in 921 bytes} [Mon Aug 1 15:16:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:13.719433 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59e930-30f8-448f-a519-f9149077b70e None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:13.721361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59e930-30f8-448f-a519-f9149077b70e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:13.723222 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59e930-30f8-448f-a519-f9149077b70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:13.807806 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-ec59e930-30f8-448f-a519-f9149077b70e None None] MFA Rules not processed for user `cd68383bd497477bb2d2292e814d799a`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:13.810739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ec59e930-30f8-448f-a519-f9149077b70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:13.845742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4833b25f-47f0-4fe9-8ad4-43882dcfe536 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d1b2f228db54737a4a7901d6f2ab7c9', 'request_id': 'req-4833b25f-47f0-4fe9-8ad4-43882dcfe536'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:13.860624 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4833b25f-47f0-4fe9-8ad4-43882dcfe536 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d1b2f228db54737a4a7901d6f2ab7c9', 'request_id': 'req-4833b25f-47f0-4fe9-8ad4-43882dcfe536'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:13.877953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4833b25f-47f0-4fe9-8ad4-43882dcfe536 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d1b2f228db54737a4a7901d6f2ab7c9', 'request_id': 'req-4833b25f-47f0-4fe9-8ad4-43882dcfe536'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:13.909646 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4833b25f-47f0-4fe9-8ad4-43882dcfe536 admin admin] The token cache is being invalidate because project 8d1b2f228db54737a4a7901d6f2ab7c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:13.910552 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4833b25f-47f0-4fe9-8ad4-43882dcfe536 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:13.913057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4833b25f-47f0-4fe9-8ad4-43882dcfe536 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:13.917509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1144/2305] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:16:13 2022] DELETE /identity/v3/projects/8d1b2f228db54737a4a7901d6f2ab7c9 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:13.920392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cc24e-1118-49dd-9f6a-f5da486b6b03 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:13.921505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cc24e-1118-49dd-9f6a-f5da486b6b03 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:13.922409 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cc24e-1118-49dd-9f6a-f5da486b6b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:13.965360 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-4a0cc24e-1118-49dd-9f6a-f5da486b6b03 None None] MFA Rules not processed for user `4eba5f64833b482d81c53721c8442970`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:13.968449 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-4a0cc24e-1118-49dd-9f6a-f5da486b6b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:14.007871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:14.049357 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1162/2306] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:14.051984 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468bb38-f755-4dd8-b5eb-e587bd021f29 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:14.052573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468bb38-f755-4dd8-b5eb-e587bd021f29 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:14.053931 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468bb38-f755-4dd8-b5eb-e587bd021f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:14.109236 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-b468bb38-f755-4dd8-b5eb-e587bd021f29 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:14.112214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b468bb38-f755-4dd8-b5eb-e587bd021f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:14.183363 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1145/2307] 10.222.0.93 () {58 vars in 1018 bytes} [Mon Aug 1 15:16:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:14.193104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cc24e-1118-49dd-9f6a-f5da486b6b03 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:14.198944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-4a0cc24e-1118-49dd-9f6a-f5da486b6b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:14.292814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:14.367045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cc24e-1118-49dd-9f6a-f5da486b6b03 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:14.370087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1163/2308] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:14.373201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468bb38-f755-4dd8-b5eb-e587bd021f29 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:14.377413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b468bb38-f755-4dd8-b5eb-e587bd021f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:14.387112 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8cdab-96d5-4606-8c5d-3095b43c16f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:14.388395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8cdab-96d5-4606-8c5d-3095b43c16f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:14.389306 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8cdab-96d5-4606-8c5d-3095b43c16f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:14.390017 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8cdab-96d5-4606-8c5d-3095b43c16f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:14.408324 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468bb38-f755-4dd8-b5eb-e587bd021f29 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:14.418127 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bed8a-3863-4d3e-91b2-8a8d44ebd77c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:14.419740 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bed8a-3863-4d3e-91b2-8a8d44ebd77c service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:14.420420 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bed8a-3863-4d3e-91b2-8a8d44ebd77c service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:14.421043 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bed8a-3863-4d3e-91b2-8a8d44ebd77c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:14.428615 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8cdab-96d5-4606-8c5d-3095b43c16f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:14.431552 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8cdab-96d5-4606-8c5d-3095b43c16f6 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:14.448819 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408bed8a-3863-4d3e-91b2-8a8d44ebd77c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:14.453393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408bed8a-3863-4d3e-91b2-8a8d44ebd77c service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:14.474550 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1146/2309] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:14.482027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848bcf8-688b-40bd-806f-8979dce44144 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:14.482815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848bcf8-688b-40bd-806f-8979dce44144 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:14.483300 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848bcf8-688b-40bd-806f-8979dce44144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:14.485137 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1164/2310] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:14.597832 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-8848bcf8-688b-40bd-806f-8979dce44144 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:14.600092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-8848bcf8-688b-40bd-806f-8979dce44144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:14.635420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1147/2311] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:14.640556 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bed8a-3863-4d3e-91b2-8a8d44ebd77c service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:14.696037 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bed8a-3863-4d3e-91b2-8a8d44ebd77c service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:14.733329 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f2103-2fe6-4332-a972-798e3ba4f7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:14.736220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f2103-2fe6-4332-a972-798e3ba4f7db admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:14.739542 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f2103-2fe6-4332-a972-798e3ba4f7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:14.742342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f2103-2fe6-4332-a972-798e3ba4f7db admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:14.743778 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f2103-2fe6-4332-a972-798e3ba4f7db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:16:14.746743 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f2103-2fe6-4332-a972-798e3ba4f7db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:14.754090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f2103-2fe6-4332-a972-798e3ba4f7db admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:14.807488 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1165/2312] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:16:14.808175 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdd0c2-51b6-4cfe-8819-840b69597d8b None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:14.808743 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdd0c2-51b6-4cfe-8819-840b69597d8b None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:14.809245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdd0c2-51b6-4cfe-8819-840b69597d8b None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:14.809505 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f2103-2fe6-4332-a972-798e3ba4f7db admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:14.811866 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1148/2313] 10.222.0.93 () {54 vars in 921 bytes} [Mon Aug 1 15:16:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:14.825976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cd040-ad92-4656-8c02-7a86d948b1e4 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:14.827343 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cd040-ad92-4656-8c02-7a86d948b1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:14.827916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cd040-ad92-4656-8c02-7a86d948b1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:14.858704 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f2103-2fe6-4332-a972-798e3ba4f7db admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:14.860081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-ee1cd040-ad92-4656-8c02-7a86d948b1e4 None None] MFA Rules not processed for user `3d9be74621d14c94b95ae94947e5abbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:14.862384 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ee1cd040-ad92-4656-8c02-7a86d948b1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:14.882255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d48a5-0606-4982-a825-a7d9e42d1c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:14.883236 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d48a5-0606-4982-a825-a7d9e42d1c49 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:14.883743 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d48a5-0606-4982-a825-a7d9e42d1c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:14.884239 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d48a5-0606-4982-a825-a7d9e42d1c49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:14.896007 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d48a5-0606-4982-a825-a7d9e42d1c49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:14.900708 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d48a5-0606-4982-a825-a7d9e42d1c49 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:14.986152 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1166/2314] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:16:14 2022] POST /identity/v3/projects => generated 379 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:14.993191 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d48a5-0606-4982-a825-a7d9e42d1c49 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:15.022712 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1149/2315] 10.222.0.93 () {58 vars in 1018 bytes} [Mon Aug 1 15:16:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:15.053549 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1cd040-ad92-4656-8c02-7a86d948b1e4 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:15.068092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1cd040-ad92-4656-8c02-7a86d948b1e4 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:15.195287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d48a5-0606-4982-a825-a7d9e42d1c49 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:15.213931 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52e03c-25fd-4aad-a0d4-9319ea4b7234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:15.214966 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52e03c-25fd-4aad-a0d4-9319ea4b7234 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.215606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52e03c-25fd-4aad-a0d4-9319ea4b7234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.216204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52e03c-25fd-4aad-a0d4-9319ea4b7234 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:15.217258 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52e03c-25fd-4aad-a0d4-9319ea4b7234 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:15.218456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c6172-7615-4d44-8c74-c86a39dd5d45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:15.220552 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c6172-7615-4d44-8c74-c86a39dd5d45 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.220552 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c6172-7615-4d44-8c74-c86a39dd5d45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.221660 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c6172-7615-4d44-8c74-c86a39dd5d45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:15.222179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52e03c-25fd-4aad-a0d4-9319ea4b7234 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:15.231849 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:15.288518 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1167/2316] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:16:14 2022] POST /identity/v3/users => generated 290 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:16:15.289412 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52e03c-25fd-4aad-a0d4-9319ea4b7234 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:15.322469 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52e03c-25fd-4aad-a0d4-9319ea4b7234 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:15.334628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf524b0-0392-49d9-81de-64bacb3ba9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:15.336150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf524b0-0392-49d9-81de-64bacb3ba9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.349385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf524b0-0392-49d9-81de-64bacb3ba9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.352613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf524b0-0392-49d9-81de-64bacb3ba9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:15.358530 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf524b0-0392-49d9-81de-64bacb3ba9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:15.392872 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf524b0-0392-49d9-81de-64bacb3ba9ee admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:15.428439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c6172-7615-4d44-8c74-c86a39dd5d45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:15.431650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c6172-7615-4d44-8c74-c86a39dd5d45 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:15.445064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1168/2317] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:15.450656 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf524b0-0392-49d9-81de-64bacb3ba9ee admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:15.457095 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1150/2318] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:16:15.481240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf524b0-0392-49d9-81de-64bacb3ba9ee admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:15.513769 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9670258-7e1e-4b8a-8e60-63e41657e227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:15.515747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9670258-7e1e-4b8a-8e60-63e41657e227 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.516219 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9670258-7e1e-4b8a-8e60-63e41657e227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.516756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9670258-7e1e-4b8a-8e60-63e41657e227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:15.527465 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9670258-7e1e-4b8a-8e60-63e41657e227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:15.530714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9670258-7e1e-4b8a-8e60-63e41657e227 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:15.575747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1169/2319] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:16:15.581918 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c6172-7615-4d44-8c74-c86a39dd5d45 service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:15.611774 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c6172-7615-4d44-8c74-c86a39dd5d45 service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:15.628741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65bf0b-b2ea-4aa8-a5ca-cedebba32c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:15.630324 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65bf0b-b2ea-4aa8-a5ca-cedebba32c35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.632371 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65bf0b-b2ea-4aa8-a5ca-cedebba32c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.633353 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65bf0b-b2ea-4aa8-a5ca-cedebba32c35 admin admin] PATH_INFO: `/v3/projects/821ce227288a424ea530e4a7c08e41c9/users/b687d38492594eab876740bdfb7db9ff/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:15.670393 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65bf0b-b2ea-4aa8-a5ca-cedebba32c35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=821ce227288a424ea530e4a7c08e41c9, user_id=b687d38492594eab876740bdfb7db9ff, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:15.678064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65bf0b-b2ea-4aa8-a5ca-cedebba32c35 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:15.710027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1151/2320] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:16:15 2022] PUT /identity/v3/projects/821ce227288a424ea530e4a7c08e41c9/users/b687d38492594eab876740bdfb7db9ff/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:15.712944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b36d0-45c3-4827-9749-6a268d6b15ab None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.713452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b36d0-45c3-4827-9749-6a268d6b15ab None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.713940 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b36d0-45c3-4827-9749-6a268d6b15ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:15.752098 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-3b7b36d0-45c3-4827-9749-6a268d6b15ab None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:15.755201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3b7b36d0-45c3-4827-9749-6a268d6b15ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:15.896529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1170/2321] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:15.907413 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65bf0b-b2ea-4aa8-a5ca-cedebba32c35 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:15.910697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-fb65bf0b-b2ea-4aa8-a5ca-cedebba32c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:16.046159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65bf0b-b2ea-4aa8-a5ca-cedebba32c35 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:16.053524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc3da3-23ce-45e0-bd53-7f5786d1d4be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:16.054837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc3da3-23ce-45e0-bd53-7f5786d1d4be service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:16.055609 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc3da3-23ce-45e0-bd53-7f5786d1d4be service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:16.056091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc3da3-23ce-45e0-bd53-7f5786d1d4be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:16.078437 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc3da3-23ce-45e0-bd53-7f5786d1d4be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:16.081546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc3da3-23ce-45e0-bd53-7f5786d1d4be service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:16.104150 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1152/2322] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:25.235200 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b184b0-634d-4ccd-8a9a-4ace3b9f1a66 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:25.235970 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b184b0-634d-4ccd-8a9a-4ace3b9f1a66 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:25.236135 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b184b0-634d-4ccd-8a9a-4ace3b9f1a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:25.249482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:25.323780 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-f7b184b0-634d-4ccd-8a9a-4ace3b9f1a66 None None] MFA Rules not processed for user `b687d38492594eab876740bdfb7db9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:25.328494 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f7b184b0-634d-4ccd-8a9a-4ace3b9f1a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:25.511115 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1171/2323] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:16:25.518432 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc3da3-23ce-45e0-bd53-7f5786d1d4be service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:25.537292 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc3da3-23ce-45e0-bd53-7f5786d1d4be service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:25.547571 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a62374-e068-47b0-987d-df765dd18928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:25.548570 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a62374-e068-47b0-987d-df765dd18928 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:25.549182 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a62374-e068-47b0-987d-df765dd18928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:25.549471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a62374-e068-47b0-987d-df765dd18928 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:25.550208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a62374-e068-47b0-987d-df765dd18928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:25.558774 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a62374-e068-47b0-987d-df765dd18928 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:25.640703 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1153/2324] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:16:25 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:25.646434 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b184b0-634d-4ccd-8a9a-4ace3b9f1a66 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:25.664984 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b184b0-634d-4ccd-8a9a-4ace3b9f1a66 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:25.822060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42805d52-52fe-4206-9f3d-d9c09068be54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:25.825416 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42805d52-52fe-4206-9f3d-d9c09068be54 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:25.826471 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42805d52-52fe-4206-9f3d-d9c09068be54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:25.827314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42805d52-52fe-4206-9f3d-d9c09068be54 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:25.829963 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42805d52-52fe-4206-9f3d-d9c09068be54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:25.836731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42805d52-52fe-4206-9f3d-d9c09068be54 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:25.889632 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1172/2325] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:16:25 2022] POST /identity/v3/users => generated 295 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:16:25.897092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a62374-e068-47b0-987d-df765dd18928 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:25.918386 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a62374-e068-47b0-987d-df765dd18928 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:25.928590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d18cc-21e0-4873-aba9-8a5d925be490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:25.930688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d18cc-21e0-4873-aba9-8a5d925be490 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:25.930688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d18cc-21e0-4873-aba9-8a5d925be490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:25.930688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d18cc-21e0-4873-aba9-8a5d925be490 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:25.932019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d18cc-21e0-4873-aba9-8a5d925be490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:25.935265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d18cc-21e0-4873-aba9-8a5d925be490 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:26.030357 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1154/2326] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:26.038395 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42805d52-52fe-4206-9f3d-d9c09068be54 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:26.065007 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42805d52-52fe-4206-9f3d-d9c09068be54 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:26.078299 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde2669-5689-4250-81f9-a9f80cd4ddd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:26.080688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde2669-5689-4250-81f9-a9f80cd4ddd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:26.081655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde2669-5689-4250-81f9-a9f80cd4ddd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:26.081867 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde2669-5689-4250-81f9-a9f80cd4ddd6 admin admin] PATH_INFO: `/v3/projects/5c98a85ab7ea46189515430e8dc1b492/users/1b2f63b200cc4208bd8df3b407eb8904/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:26.153762 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde2669-5689-4250-81f9-a9f80cd4ddd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c98a85ab7ea46189515430e8dc1b492, user_id=1b2f63b200cc4208bd8df3b407eb8904, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:26.159525 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde2669-5689-4250-81f9-a9f80cd4ddd6 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:26.206448 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7cfcf-566f-4f0b-9889-697345a9d921 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:26.207912 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7cfcf-566f-4f0b-9889-697345a9d921 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:26.208779 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7cfcf-566f-4f0b-9889-697345a9d921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:26.213367 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1173/2327] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:16:26 2022] PUT /identity/v3/projects/5c98a85ab7ea46189515430e8dc1b492/users/1b2f63b200cc4208bd8df3b407eb8904/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:16:26.219479 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde2669-5689-4250-81f9-a9f80cd4ddd6 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:26.269643 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-d8b7cfcf-566f-4f0b-9889-697345a9d921 None None] MFA Rules not processed for user `4370f8f474c2446e989d0d8c5b75e87a`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:26.274447 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d8b7cfcf-566f-4f0b-9889-697345a9d921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:26.407211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:26.460565 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:26.469583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde2669-5689-4250-81f9-a9f80cd4ddd6 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:26.493612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1155/2328] 10.222.0.93 () {58 vars in 1009 bytes} [Mon Aug 1 15:16:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:16:26.496999 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b08a7a-6c3f-4b24-8735-da0ae53c9c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:26.498117 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b08a7a-6c3f-4b24-8735-da0ae53c9c47 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:26.498657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b08a7a-6c3f-4b24-8735-da0ae53c9c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:26.499133 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b08a7a-6c3f-4b24-8735-da0ae53c9c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:26.505212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b08a7a-6c3f-4b24-8735-da0ae53c9c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:26.511161 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b08a7a-6c3f-4b24-8735-da0ae53c9c47 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:26.523046 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7cfcf-566f-4f0b-9889-697345a9d921 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:26.525623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d8b7cfcf-566f-4f0b-9889-697345a9d921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:26.554539 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1174/2329] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:16:26.561323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b08a7a-6c3f-4b24-8735-da0ae53c9c47 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:26.587279 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b08a7a-6c3f-4b24-8735-da0ae53c9c47 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:26.596327 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da806e-cadd-4a53-a43c-89452bf2e14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:26.597698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da806e-cadd-4a53-a43c-89452bf2e14b admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:26.598358 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da806e-cadd-4a53-a43c-89452bf2e14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:26.598996 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da806e-cadd-4a53-a43c-89452bf2e14b admin admin] PATH_INFO: `/v3/projects/5c98a85ab7ea46189515430e8dc1b492/users/1b2f63b200cc4208bd8df3b407eb8904/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:26.603378 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da806e-cadd-4a53-a43c-89452bf2e14b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c98a85ab7ea46189515430e8dc1b492, user_id=1b2f63b200cc4208bd8df3b407eb8904, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:26.609337 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da806e-cadd-4a53-a43c-89452bf2e14b admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:26.656326 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1175/2330] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:16:26 2022] PUT /identity/v3/projects/5c98a85ab7ea46189515430e8dc1b492/users/1b2f63b200cc4208bd8df3b407eb8904/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:26.725124 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7cfcf-566f-4f0b-9889-697345a9d921 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:26.992055 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863009a6-ca1c-41f0-a180-a015a8acfc81 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4df37b02152c48509243b73ebbf49d8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:26.994695 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863009a6-ca1c-41f0-a180-a015a8acfc81 service swift] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:26.995318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863009a6-ca1c-41f0-a180-a015a8acfc81 service swift] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:26.995984 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863009a6-ca1c-41f0-a180-a015a8acfc81 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:27.184744 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863009a6-ca1c-41f0-a180-a015a8acfc81 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:27.189020 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863009a6-ca1c-41f0-a180-a015a8acfc81 service swift] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:27.247246 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1156/2331] 10.222.0.93 () {60 vars in 1384 bytes} [Mon Aug 1 15:16:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 725 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 15:16:36.635410 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=8,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:37.226714 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=88 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:40.970766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da806e-cadd-4a53-a43c-89452bf2e14b admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:40.975373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-68da806e-cadd-4a53-a43c-89452bf2e14b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:41.215520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da806e-cadd-4a53-a43c-89452bf2e14b admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:41.237103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dffcd8-3055-41fb-af16-d2c6764da2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:41.238411 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dffcd8-3055-41fb-af16-d2c6764da2b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:41.238707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dffcd8-3055-41fb-af16-d2c6764da2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:41.238983 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dffcd8-3055-41fb-af16-d2c6764da2b6 admin admin] PATH_INFO: `/v3/users/cd68383bd497477bb2d2292e814d799a` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:41.249985 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dffcd8-3055-41fb-af16-d2c6764da2b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd68383bd497477bb2d2292e814d799a)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:41.259236 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dffcd8-3055-41fb-af16-d2c6764da2b6 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:41.464910 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a9dffcd8-3055-41fb-af16-d2c6764da2b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd68383bd497477bb2d2292e814d799a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:41.501784 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a9dffcd8-3055-41fb-af16-d2c6764da2b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd68383bd497477bb2d2292e814d799a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:41.535303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a9dffcd8-3055-41fb-af16-d2c6764da2b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd68383bd497477bb2d2292e814d799a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:41.536275 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a9dffcd8-3055-41fb-af16-d2c6764da2b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd68383bd497477bb2d2292e814d799a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:41.537248 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a9dffcd8-3055-41fb-af16-d2c6764da2b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd68383bd497477bb2d2292e814d799a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:41.634098 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1176/2332] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:16:40 2022] DELETE /identity/v3/users/cd68383bd497477bb2d2292e814d799a => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:41.640456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fdd176-48a5-4a02-8ce0-9a5b30b9edb5 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:41.641425 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fdd176-48a5-4a02-8ce0-9a5b30b9edb5 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:41.641425 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fdd176-48a5-4a02-8ce0-9a5b30b9edb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:41.697681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-00fdd176-48a5-4a02-8ce0-9a5b30b9edb5 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:41.700780 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-00fdd176-48a5-4a02-8ce0-9a5b30b9edb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:42.062385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f05c8-aced-410f-b4b3-b4778696d477 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:42.063346 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f05c8-aced-410f-b4b3-b4778696d477 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:42.063346 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f05c8-aced-410f-b4b3-b4778696d477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:42.094574 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1157/2333] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:42.115343 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fdd176-48a5-4a02-8ce0-9a5b30b9edb5 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:42.136259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-00fdd176-48a5-4a02-8ce0-9a5b30b9edb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:42.184276 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-1f3f05c8-aced-410f-b4b3-b4778696d477 None None] MFA Rules not processed for user `1b2f63b200cc4208bd8df3b407eb8904`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:42.195703 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-1f3f05c8-aced-410f-b4b3-b4778696d477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:42.545842 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fdd176-48a5-4a02-8ce0-9a5b30b9edb5 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:42.557987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6ce4b-73cb-4402-8914-88dd59645d58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:42.559820 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6ce4b-73cb-4402-8914-88dd59645d58 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:42.560784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6ce4b-73cb-4402-8914-88dd59645d58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:42.561578 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6ce4b-73cb-4402-8914-88dd59645d58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:42.578363 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1177/2334] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 516 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:16:42.597721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6ce4b-73cb-4402-8914-88dd59645d58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:42.601686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6ce4b-73cb-4402-8914-88dd59645d58 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:42.610212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f05c8-aced-410f-b4b3-b4778696d477 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:42.614472 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-1f3f05c8-aced-410f-b4b3-b4778696d477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:42.630261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:42.643375 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1158/2335] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:42.816861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f05c8-aced-410f-b4b3-b4778696d477 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:42.837503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cedba7a-7bef-40eb-9861-a0b574997c06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:42.838728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedba7a-7bef-40eb-9861-a0b574997c06 service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:42.839217 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedba7a-7bef-40eb-9861-a0b574997c06 service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:42.840568 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedba7a-7bef-40eb-9861-a0b574997c06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:42.845216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-7cedba7a-7bef-40eb-9861-a0b574997c06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:43.002423 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cedba7a-7bef-40eb-9861-a0b574997c06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:43.011566 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cedba7a-7bef-40eb-9861-a0b574997c06 service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:43.038916 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1178/2336] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:16:43.391140 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6ce4b-73cb-4402-8914-88dd59645d58 service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:43.400503 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-7bb6ce4b-73cb-4402-8914-88dd59645d58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:43.453911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6ce4b-73cb-4402-8914-88dd59645d58 service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:43.477891 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee8ca6-7452-422b-bfb2-9bfa1b798a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:43.484139 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee8ca6-7452-422b-bfb2-9bfa1b798a5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:43.485090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee8ca6-7452-422b-bfb2-9bfa1b798a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:43.485885 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee8ca6-7452-422b-bfb2-9bfa1b798a5c admin admin] PATH_INFO: `/v3/projects/2842c4382a22430497f28e814c59560a` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:43.492520 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee8ca6-7452-422b-bfb2-9bfa1b798a5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2842c4382a22430497f28e814c59560a)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:43.498326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee8ca6-7452-422b-bfb2-9bfa1b798a5c admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:43.732987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9dee8ca6-7452-422b-bfb2-9bfa1b798a5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2842c4382a22430497f28e814c59560a', 'request_id': 'req-9dee8ca6-7452-422b-bfb2-9bfa1b798a5c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:43.759301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9dee8ca6-7452-422b-bfb2-9bfa1b798a5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2842c4382a22430497f28e814c59560a', 'request_id': 'req-9dee8ca6-7452-422b-bfb2-9bfa1b798a5c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:43.784274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9dee8ca6-7452-422b-bfb2-9bfa1b798a5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2842c4382a22430497f28e814c59560a', 'request_id': 'req-9dee8ca6-7452-422b-bfb2-9bfa1b798a5c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:43.821483 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9dee8ca6-7452-422b-bfb2-9bfa1b798a5c admin admin] The token cache is being invalidate because project 2842c4382a22430497f28e814c59560a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:43.823027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9dee8ca6-7452-422b-bfb2-9bfa1b798a5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:43.825421 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9dee8ca6-7452-422b-bfb2-9bfa1b798a5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:43.833050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1159/2337] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:16:43 2022] DELETE /identity/v3/projects/2842c4382a22430497f28e814c59560a => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:43.841797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda3eb3-7b05-4050-ba48-707b55be0dfa None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:43.842761 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda3eb3-7b05-4050-ba48-707b55be0dfa None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:43.843032 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda3eb3-7b05-4050-ba48-707b55be0dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:43.902947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-5eda3eb3-7b05-4050-ba48-707b55be0dfa None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:43.908274 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5eda3eb3-7b05-4050-ba48-707b55be0dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:44.171350 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1179/2338] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:44.187394 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d17ad-46f3-40ee-8c14-7fde1c8d4ba4 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.189301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d17ad-46f3-40ee-8c14-7fde1c8d4ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.190622 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d17ad-46f3-40ee-8c14-7fde1c8d4ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:44.235892 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-943d17ad-46f3-40ee-8c14-7fde1c8d4ba4 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:44.238905 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-943d17ad-46f3-40ee-8c14-7fde1c8d4ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:44.257412 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1160/2339] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:44.264540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda3eb3-7b05-4050-ba48-707b55be0dfa None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:44.285951 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda3eb3-7b05-4050-ba48-707b55be0dfa None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:44.301432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df481d2a-7270-4999-b18f-1fbb87b61342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:44.302790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df481d2a-7270-4999-b18f-1fbb87b61342 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.303280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df481d2a-7270-4999-b18f-1fbb87b61342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.303776 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df481d2a-7270-4999-b18f-1fbb87b61342 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:44.305613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df481d2a-7270-4999-b18f-1fbb87b61342 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:16:44.306011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df481d2a-7270-4999-b18f-1fbb87b61342 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:44.310136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df481d2a-7270-4999-b18f-1fbb87b61342 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:44.361704 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1180/2340] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:16:44.368990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d17ad-46f3-40ee-8c14-7fde1c8d4ba4 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:44.400632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d17ad-46f3-40ee-8c14-7fde1c8d4ba4 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:44.410255 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d7b88-0df6-4ceb-b840-b0d910daa097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:44.410255 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d7b88-0df6-4ceb-b840-b0d910daa097 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.410255 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d7b88-0df6-4ceb-b840-b0d910daa097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.410255 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d7b88-0df6-4ceb-b840-b0d910daa097 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:44.410958 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d7b88-0df6-4ceb-b840-b0d910daa097 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:44.412104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d7b88-0df6-4ceb-b840-b0d910daa097 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:44.464310 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:44.493334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1161/2341] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:16:44 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:44.494907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df481d2a-7270-4999-b18f-1fbb87b61342 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:44.533582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df481d2a-7270-4999-b18f-1fbb87b61342 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:44.597946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:44.669071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff0393-9349-4cbb-ba6d-a25a0f2e843c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:44.670087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff0393-9349-4cbb-ba6d-a25a0f2e843c admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.670521 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff0393-9349-4cbb-ba6d-a25a0f2e843c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.671010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff0393-9349-4cbb-ba6d-a25a0f2e843c admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:44.672368 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff0393-9349-4cbb-ba6d-a25a0f2e843c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:44.677360 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff0393-9349-4cbb-ba6d-a25a0f2e843c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:44.731530 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1181/2342] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:16:44 2022] POST /identity/v3/users => generated 286 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:16:44.741992 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d7b88-0df6-4ceb-b840-b0d910daa097 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:44.768948 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d7b88-0df6-4ceb-b840-b0d910daa097 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:44.779531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e7afa-6a9d-4a5b-95a7-8bfa4839e7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:44.781806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e7afa-6a9d-4a5b-95a7-8bfa4839e7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.782292 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e7afa-6a9d-4a5b-95a7-8bfa4839e7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.782781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e7afa-6a9d-4a5b-95a7-8bfa4839e7c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:44.784137 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e7afa-6a9d-4a5b-95a7-8bfa4839e7c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:44.787680 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e7afa-6a9d-4a5b-95a7-8bfa4839e7c2 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:44.809691 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1162/2343] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:44.817563 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff0393-9349-4cbb-ba6d-a25a0f2e843c admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:44.841758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff0393-9349-4cbb-ba6d-a25a0f2e843c admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:44.852753 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7013c-4e01-47fa-98bb-d6483f0c711b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:44.854164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7013c-4e01-47fa-98bb-d6483f0c711b admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.854464 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7013c-4e01-47fa-98bb-d6483f0c711b admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.855228 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7013c-4e01-47fa-98bb-d6483f0c711b admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:44.857220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7013c-4e01-47fa-98bb-d6483f0c711b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:44.860251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7013c-4e01-47fa-98bb-d6483f0c711b admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:44.886572 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1182/2344] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:44.893753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e7afa-6a9d-4a5b-95a7-8bfa4839e7c2 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:44.907436 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e7afa-6a9d-4a5b-95a7-8bfa4839e7c2 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:44.915466 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286b45b-e3df-48ae-85b3-e6e0a2f68b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:44.917249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286b45b-e3df-48ae-85b3-e6e0a2f68b44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.917718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286b45b-e3df-48ae-85b3-e6e0a2f68b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.918129 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286b45b-e3df-48ae-85b3-e6e0a2f68b44 admin admin] PATH_INFO: `/v3/projects/31c3f6e135bd43c087ca519efbba220f/users/956fdcd7e7a64721aaa4c5e150d712de/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:44.952014 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286b45b-e3df-48ae-85b3-e6e0a2f68b44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31c3f6e135bd43c087ca519efbba220f, user_id=956fdcd7e7a64721aaa4c5e150d712de, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:44.956756 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286b45b-e3df-48ae-85b3-e6e0a2f68b44 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:44.983274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1163/2345] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:16:44 2022] PUT /identity/v3/projects/31c3f6e135bd43c087ca519efbba220f/users/956fdcd7e7a64721aaa4c5e150d712de/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:45.089718 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acb1df-4ede-4080-ae47-e66487787b6b None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:45.090584 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acb1df-4ede-4080-ae47-e66487787b6b None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:45.091321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acb1df-4ede-4080-ae47-e66487787b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:45.155732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-57acb1df-4ede-4080-ae47-e66487787b6b None None] MFA Rules not processed for user `956fdcd7e7a64721aaa4c5e150d712de`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:45.157876 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-57acb1df-4ede-4080-ae47-e66487787b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:45.324164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1183/2346] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:16:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:45.349667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286b45b-e3df-48ae-85b3-e6e0a2f68b44 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:45.353521 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-5286b45b-e3df-48ae-85b3-e6e0a2f68b44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:45.570616 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286b45b-e3df-48ae-85b3-e6e0a2f68b44 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:45.584084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012b1b4-ef1c-4282-a56c-fea48b88a92e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4461a389da5848f5afd9854758707829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:45.585859 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012b1b4-ef1c-4282-a56c-fea48b88a92e service glance] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:45.586413 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012b1b4-ef1c-4282-a56c-fea48b88a92e service glance] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:45.586892 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012b1b4-ef1c-4282-a56c-fea48b88a92e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:45.607700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012b1b4-ef1c-4282-a56c-fea48b88a92e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:45.610640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012b1b4-ef1c-4282-a56c-fea48b88a92e service glance] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:45.637744 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1164/2347] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:55.315414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:55.621433 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:04.445403 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b447ddc-5514-4a43-8c06-1778fc67eb47 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:04.446978 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b447ddc-5514-4a43-8c06-1778fc67eb47 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:04.446978 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b447ddc-5514-4a43-8c06-1778fc67eb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:04.472891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-4b447ddc-5514-4a43-8c06-1778fc67eb47 None None] MFA Rules not processed for user `4370f8f474c2446e989d0d8c5b75e87a`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:04.477567 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4b447ddc-5514-4a43-8c06-1778fc67eb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:04.566017 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1184/2348] 10.222.0.93 () {58 vars in 967 bytes} [Mon Aug 1 15:17:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:04.593072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012b1b4-ef1c-4282-a56c-fea48b88a92e service glance] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:04.595418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-5012b1b4-ef1c-4282-a56c-fea48b88a92e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:04.694642 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012b1b4-ef1c-4282-a56c-fea48b88a92e service glance] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:04.699996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65844f3f-d1a4-4703-a5f2-fbf4fdbc2487 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4df37b02152c48509243b73ebbf49d8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:04.700767 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65844f3f-d1a4-4703-a5f2-fbf4fdbc2487 service swift] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:04.700996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65844f3f-d1a4-4703-a5f2-fbf4fdbc2487 service swift] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:04.701264 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65844f3f-d1a4-4703-a5f2-fbf4fdbc2487 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:04.714067 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65844f3f-d1a4-4703-a5f2-fbf4fdbc2487 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:04.715738 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65844f3f-d1a4-4703-a5f2-fbf4fdbc2487 service swift] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:04.725413 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1165/2349] 10.222.0.93 () {60 vars in 1384 bytes} [Mon Aug 1 15:17:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:17:13.366367 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9c61c-7bce-4dea-8e27-69350142562b None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:13.367064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9c61c-7bce-4dea-8e27-69350142562b None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:13.367064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9c61c-7bce-4dea-8e27-69350142562b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:13.387457 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-d9d9c61c-7bce-4dea-8e27-69350142562b None None] MFA Rules not processed for user `b687d38492594eab876740bdfb7db9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:13.388728 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-d9d9c61c-7bce-4dea-8e27-69350142562b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:13.472660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1185/2350] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:17:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:14.721727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=32 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:23.464366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=50 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:25.516566 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65844f3f-d1a4-4703-a5f2-fbf4fdbc2487 service swift] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:25.640107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65844f3f-d1a4-4703-a5f2-fbf4fdbc2487 service swift] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:25.646196 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d398b-de51-4bcf-a0aa-ccff8be1b0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:25.646867 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d398b-de51-4bcf-a0aa-ccff8be1b0bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:25.647148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d398b-de51-4bcf-a0aa-ccff8be1b0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:25.647655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d398b-de51-4bcf-a0aa-ccff8be1b0bd admin admin] PATH_INFO: `/v3/users/956fdcd7e7a64721aaa4c5e150d712de` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:25.649005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d398b-de51-4bcf-a0aa-ccff8be1b0bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=956fdcd7e7a64721aaa4c5e150d712de)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:25.651013 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d398b-de51-4bcf-a0aa-ccff8be1b0bd admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:25.728603 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-da1d398b-de51-4bcf-a0aa-ccff8be1b0bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '956fdcd7e7a64721aaa4c5e150d712de'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:25.729580 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-da1d398b-de51-4bcf-a0aa-ccff8be1b0bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '956fdcd7e7a64721aaa4c5e150d712de'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:25.743745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-da1d398b-de51-4bcf-a0aa-ccff8be1b0bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '956fdcd7e7a64721aaa4c5e150d712de'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:25.744644 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-da1d398b-de51-4bcf-a0aa-ccff8be1b0bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '956fdcd7e7a64721aaa4c5e150d712de'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:25.763109 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-da1d398b-de51-4bcf-a0aa-ccff8be1b0bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '956fdcd7e7a64721aaa4c5e150d712de'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:25.781514 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1166/2351] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:17:25 2022] DELETE /identity/v3/users/956fdcd7e7a64721aaa4c5e150d712de => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:25.787219 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc2fd4-28b0-4c20-b206-eac0d41ddac6 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:25.788162 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc2fd4-28b0-4c20-b206-eac0d41ddac6 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:25.788162 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc2fd4-28b0-4c20-b206-eac0d41ddac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:25.819219 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-88dc2fd4-28b0-4c20-b206-eac0d41ddac6 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:25.821476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-88dc2fd4-28b0-4c20-b206-eac0d41ddac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:25.912784 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1186/2352] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:17:25.931385 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d398b-de51-4bcf-a0aa-ccff8be1b0bd admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:25.934002 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-da1d398b-de51-4bcf-a0aa-ccff8be1b0bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:26.030177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d398b-de51-4bcf-a0aa-ccff8be1b0bd admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:26.037583 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2446fde-6bd7-4758-8a87-86bb7144407a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:26.038240 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2446fde-6bd7-4758-8a87-86bb7144407a service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:26.038515 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2446fde-6bd7-4758-8a87-86bb7144407a service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:26.038798 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2446fde-6bd7-4758-8a87-86bb7144407a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:26.052372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2446fde-6bd7-4758-8a87-86bb7144407a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:26.054548 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2446fde-6bd7-4758-8a87-86bb7144407a service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:26.072573 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1167/2353] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:26.585033 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc2fd4-28b0-4c20-b206-eac0d41ddac6 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:26.587657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-88dc2fd4-28b0-4c20-b206-eac0d41ddac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:26.612332 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc2fd4-28b0-4c20-b206-eac0d41ddac6 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:26.622944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd84608-f0e1-4782-9ab8-84d1475ad37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:26.624039 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd84608-f0e1-4782-9ab8-84d1475ad37b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:26.624564 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd84608-f0e1-4782-9ab8-84d1475ad37b admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:26.625016 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd84608-f0e1-4782-9ab8-84d1475ad37b admin admin] PATH_INFO: `/v3/projects/31c3f6e135bd43c087ca519efbba220f` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:26.627477 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd84608-f0e1-4782-9ab8-84d1475ad37b admin admin] RBAC: Authorizing `identity:delete_project(project_id=31c3f6e135bd43c087ca519efbba220f)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:26.631489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd84608-f0e1-4782-9ab8-84d1475ad37b admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:26.729118 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4bd84608-f0e1-4782-9ab8-84d1475ad37b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31c3f6e135bd43c087ca519efbba220f', 'request_id': 'req-4bd84608-f0e1-4782-9ab8-84d1475ad37b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:26.740084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4bd84608-f0e1-4782-9ab8-84d1475ad37b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31c3f6e135bd43c087ca519efbba220f', 'request_id': 'req-4bd84608-f0e1-4782-9ab8-84d1475ad37b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:26.750159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4bd84608-f0e1-4782-9ab8-84d1475ad37b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31c3f6e135bd43c087ca519efbba220f', 'request_id': 'req-4bd84608-f0e1-4782-9ab8-84d1475ad37b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:26.773387 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4bd84608-f0e1-4782-9ab8-84d1475ad37b admin admin] The token cache is being invalidate because project 31c3f6e135bd43c087ca519efbba220f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:17:26.773843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4bd84608-f0e1-4782-9ab8-84d1475ad37b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:26.775755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4bd84608-f0e1-4782-9ab8-84d1475ad37b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:26.778540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1187/2354] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:17:26 2022] DELETE /identity/v3/projects/31c3f6e135bd43c087ca519efbba220f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:36.065609 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:36.768143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:37.369235 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2446fde-6bd7-4758-8a87-86bb7144407a service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:37.378624 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2446fde-6bd7-4758-8a87-86bb7144407a service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:37.429696 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642f129-2887-4021-9f52-591bf8c9f604 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:37.430404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642f129-2887-4021-9f52-591bf8c9f604 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:37.430682 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642f129-2887-4021-9f52-591bf8c9f604 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:37.430956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642f129-2887-4021-9f52-591bf8c9f604 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:37.432766 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a642f129-2887-4021-9f52-591bf8c9f604 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:37.520287 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642f129-2887-4021-9f52-591bf8c9f604 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:37.521900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642f129-2887-4021-9f52-591bf8c9f604 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:37.533868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1168/2355] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:37.870939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63548c13-1a1b-4923-a3e4-cfa3b7d3be5c None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:37.871491 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63548c13-1a1b-4923-a3e4-cfa3b7d3be5c None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:37.871491 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63548c13-1a1b-4923-a3e4-cfa3b7d3be5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:37.891514 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-63548c13-1a1b-4923-a3e4-cfa3b7d3be5c None None] MFA Rules not processed for user `3af3a5c1c2f94e51bee936b4762dc14c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:37.892775 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-63548c13-1a1b-4923-a3e4-cfa3b7d3be5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:37.951459 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1188/2356] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:17:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:40.575738 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642f129-2887-4021-9f52-591bf8c9f604 service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:40.577724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a642f129-2887-4021-9f52-591bf8c9f604 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:40.590797 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642f129-2887-4021-9f52-591bf8c9f604 service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:40.595666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08690e36-dd5b-42da-95e7-16481ed02d01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:40.596603 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08690e36-dd5b-42da-95e7-16481ed02d01 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:40.596887 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08690e36-dd5b-42da-95e7-16481ed02d01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:40.597161 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08690e36-dd5b-42da-95e7-16481ed02d01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:40.598978 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-08690e36-dd5b-42da-95e7-16481ed02d01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:40.673023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08690e36-dd5b-42da-95e7-16481ed02d01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:40.674496 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08690e36-dd5b-42da-95e7-16481ed02d01 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:40.690885 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1169/2357] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:40 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:17:43.214051 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63548c13-1a1b-4923-a3e4-cfa3b7d3be5c None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:43.224711 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63548c13-1a1b-4923-a3e4-cfa3b7d3be5c None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:43.229270 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23231d-7f0e-4c53-8e43-e6dfd6662940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:43.230014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23231d-7f0e-4c53-8e43-e6dfd6662940 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:43.230591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23231d-7f0e-4c53-8e43-e6dfd6662940 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:43.230866 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23231d-7f0e-4c53-8e43-e6dfd6662940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:43.232976 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-bb23231d-7f0e-4c53-8e43-e6dfd6662940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:43.248726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23231d-7f0e-4c53-8e43-e6dfd6662940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:43.250334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23231d-7f0e-4c53-8e43-e6dfd6662940 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:43.263616 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1189/2358] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:50.680268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=84 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:53.258828 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=31 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:55.864348 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35078c8-e8d6-498a-b55c-1dff12bcf192 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:55.865026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35078c8-e8d6-498a-b55c-1dff12bcf192 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:55.865026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35078c8-e8d6-498a-b55c-1dff12bcf192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:55.884670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-e35078c8-e8d6-498a-b55c-1dff12bcf192 None None] MFA Rules not processed for user `3c18437883ce42369c7563224c208523`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:55.885940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e35078c8-e8d6-498a-b55c-1dff12bcf192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:55.942854 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1170/2359] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:17:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:02.225558 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23231d-7f0e-4c53-8e43-e6dfd6662940 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:02.227136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-bb23231d-7f0e-4c53-8e43-e6dfd6662940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:02.246467 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23231d-7f0e-4c53-8e43-e6dfd6662940 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:02.255941 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a1749-a1a7-4ebd-ba3c-35c03b3ab3eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:02.256974 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a1749-a1a7-4ebd-ba3c-35c03b3ab3eb service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:02.257420 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a1749-a1a7-4ebd-ba3c-35c03b3ab3eb service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:02.258461 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a1749-a1a7-4ebd-ba3c-35c03b3ab3eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:02.260666 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-512a1749-a1a7-4ebd-ba3c-35c03b3ab3eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:02.275670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a1749-a1a7-4ebd-ba3c-35c03b3ab3eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:02.277300 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a1749-a1a7-4ebd-ba3c-35c03b3ab3eb service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:02.294186 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1190/2360] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:18:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:18:04.492981 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35078c8-e8d6-498a-b55c-1dff12bcf192 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:04.494562 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e35078c8-e8d6-498a-b55c-1dff12bcf192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:04.516636 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35078c8-e8d6-498a-b55c-1dff12bcf192 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:04.520947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aace2ed-a36d-4358-b634-dee95eb94f2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:04.521587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aace2ed-a36d-4358-b634-dee95eb94f2e service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:04.521950 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aace2ed-a36d-4358-b634-dee95eb94f2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:04.522139 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aace2ed-a36d-4358-b634-dee95eb94f2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:04.524112 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-6aace2ed-a36d-4358-b634-dee95eb94f2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:04.597726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aace2ed-a36d-4358-b634-dee95eb94f2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:04.599190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aace2ed-a36d-4358-b634-dee95eb94f2e service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:04.611089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1171/2361] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:18:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:18:05.746192 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b10d8-da3d-4d55-9bb5-0743c824bec4 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:05.746816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b10d8-da3d-4d55-9bb5-0743c824bec4 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:05.746816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b10d8-da3d-4d55-9bb5-0743c824bec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:05.768398 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-2d1b10d8-da3d-4d55-9bb5-0743c824bec4 None None] MFA Rules not processed for user `5a49850f6d8a474d83821f5f49dff06c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:05.769602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-2d1b10d8-da3d-4d55-9bb5-0743c824bec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:05.777137 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1191/2362] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:18:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:18:14.604831 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=53 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:15.763134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=9 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:23.009573 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e7f6c-1b6d-4e37-a43a-86dcca61236f None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:23.010099 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e7f6c-1b6d-4e37-a43a-86dcca61236f None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:23.010226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e7f6c-1b6d-4e37-a43a-86dcca61236f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:23.030755 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-665e7f6c-1b6d-4e37-a43a-86dcca61236f None None] MFA Rules not processed for user `4370f8f474c2446e989d0d8c5b75e87a`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:23.032618 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-665e7f6c-1b6d-4e37-a43a-86dcca61236f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:23.096339 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1172/2363] 10.222.0.93 () {58 vars in 967 bytes} [Mon Aug 1 15:18:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:23.149327 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b10d8-da3d-4d55-9bb5-0743c824bec4 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:23.151085 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-2d1b10d8-da3d-4d55-9bb5-0743c824bec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:23.235457 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b10d8-da3d-4d55-9bb5-0743c824bec4 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:23.240972 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2b467-2a10-4b34-97e1-a0869b699757 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4df37b02152c48509243b73ebbf49d8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:23.241602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2b467-2a10-4b34-97e1-a0869b699757 service swift] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:23.241875 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2b467-2a10-4b34-97e1-a0869b699757 service swift] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:23.242144 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2b467-2a10-4b34-97e1-a0869b699757 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:23.255670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2b467-2a10-4b34-97e1-a0869b699757 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:23.258371 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2b467-2a10-4b34-97e1-a0869b699757 service swift] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:23.269408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1192/2364] 10.222.0.93 () {60 vars in 1384 bytes} [Mon Aug 1 15:18:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:18:24.466441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e7f6c-1b6d-4e37-a43a-86dcca61236f None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:24.476443 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e7f6c-1b6d-4e37-a43a-86dcca61236f None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:24.481004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029387cf-8241-4fae-a4ef-d3748235b74e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:24.481702 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029387cf-8241-4fae-a4ef-d3748235b74e service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:24.482049 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029387cf-8241-4fae-a4ef-d3748235b74e service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:24.482330 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029387cf-8241-4fae-a4ef-d3748235b74e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:24.484110 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-029387cf-8241-4fae-a4ef-d3748235b74e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:24.495009 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029387cf-8241-4fae-a4ef-d3748235b74e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:24.496453 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029387cf-8241-4fae-a4ef-d3748235b74e service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:24.511545 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1173/2365] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:18:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:18:25.331528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2b467-2a10-4b34-97e1-a0869b699757 service swift] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:25.333010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e0f2b467-2a10-4b34-97e1-a0869b699757 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:25.394702 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2b467-2a10-4b34-97e1-a0869b699757 service swift] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:25.398910 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8169e5-fd90-4173-8b52-e853c49d83a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:25.399533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8169e5-fd90-4173-8b52-e853c49d83a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:25.399813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8169e5-fd90-4173-8b52-e853c49d83a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:25.400087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8169e5-fd90-4173-8b52-e853c49d83a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:25.401866 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-9f8169e5-fd90-4173-8b52-e853c49d83a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:25.414649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8169e5-fd90-4173-8b52-e853c49d83a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:25.416097 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8169e5-fd90-4173-8b52-e853c49d83a4 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:25.427124 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1193/2366] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:18:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:18:30.792566 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029387cf-8241-4fae-a4ef-d3748235b74e service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:30.794734 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-029387cf-8241-4fae-a4ef-d3748235b74e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:30.881275 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029387cf-8241-4fae-a4ef-d3748235b74e service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:30.885899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1440652-bf13-4fb5-a7e2-5fe4e515a1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:30.886555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1440652-bf13-4fb5-a7e2-5fe4e515a1eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:30.887165 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1440652-bf13-4fb5-a7e2-5fe4e515a1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:30.887165 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1440652-bf13-4fb5-a7e2-5fe4e515a1eb admin admin] PATH_INFO: `/v3/users/16f05877e10442d2a8938f6805eff842` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:30.888740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1440652-bf13-4fb5-a7e2-5fe4e515a1eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=16f05877e10442d2a8938f6805eff842)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:30.890301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1440652-bf13-4fb5-a7e2-5fe4e515a1eb admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:30.944717 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-d1440652-bf13-4fb5-a7e2-5fe4e515a1eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16f05877e10442d2a8938f6805eff842'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:30.945457 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-d1440652-bf13-4fb5-a7e2-5fe4e515a1eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16f05877e10442d2a8938f6805eff842'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:30.955227 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-d1440652-bf13-4fb5-a7e2-5fe4e515a1eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16f05877e10442d2a8938f6805eff842'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:30.955886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-d1440652-bf13-4fb5-a7e2-5fe4e515a1eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16f05877e10442d2a8938f6805eff842'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:30.971313 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-d1440652-bf13-4fb5-a7e2-5fe4e515a1eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16f05877e10442d2a8938f6805eff842'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:30.986389 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1174/2367] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:18:30 2022] DELETE /identity/v3/users/16f05877e10442d2a8938f6805eff842 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:18:31.229923 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8169e5-fd90-4173-8b52-e853c49d83a4 service cinder] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:31.242839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8169e5-fd90-4173-8b52-e853c49d83a4 service cinder] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:31.323482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099df3f7-7eb3-43d1-b66f-b018345a5085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:31.324156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099df3f7-7eb3-43d1-b66f-b018345a5085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:31.324444 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099df3f7-7eb3-43d1-b66f-b018345a5085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:31.324754 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099df3f7-7eb3-43d1-b66f-b018345a5085 admin admin] PATH_INFO: `/v3/projects/03ff255ce0464ad49313432927bb4daf` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:31.326247 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099df3f7-7eb3-43d1-b66f-b018345a5085 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03ff255ce0464ad49313432927bb4daf)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:31.328963 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099df3f7-7eb3-43d1-b66f-b018345a5085 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:31.370601 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:31.384323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-099df3f7-7eb3-43d1-b66f-b018345a5085 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03ff255ce0464ad49313432927bb4daf', 'request_id': 'req-099df3f7-7eb3-43d1-b66f-b018345a5085'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:31.394785 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-099df3f7-7eb3-43d1-b66f-b018345a5085 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03ff255ce0464ad49313432927bb4daf', 'request_id': 'req-099df3f7-7eb3-43d1-b66f-b018345a5085'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:31.400961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-099df3f7-7eb3-43d1-b66f-b018345a5085 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03ff255ce0464ad49313432927bb4daf', 'request_id': 'req-099df3f7-7eb3-43d1-b66f-b018345a5085'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:31.414221 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-099df3f7-7eb3-43d1-b66f-b018345a5085 admin admin] The token cache is being invalidate because project 03ff255ce0464ad49313432927bb4daf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:18:31.414543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-099df3f7-7eb3-43d1-b66f-b018345a5085 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:31.415168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-099df3f7-7eb3-43d1-b66f-b018345a5085 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:31.416715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1194/2368] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:18:31 2022] DELETE /identity/v3/projects/03ff255ce0464ad49313432927bb4daf => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:18:31.421211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1440652-bf13-4fb5-a7e2-5fe4e515a1eb admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:31.422738 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d1440652-bf13-4fb5-a7e2-5fe4e515a1eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:31.437050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1440652-bf13-4fb5-a7e2-5fe4e515a1eb admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:31.446766 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:31.514069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9c1b1-b1d4-422a-a4a7-40e4cc932681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:31.514731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9c1b1-b1d4-422a-a4a7-40e4cc932681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:31.515043 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9c1b1-b1d4-422a-a4a7-40e4cc932681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:31.515326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9c1b1-b1d4-422a-a4a7-40e4cc932681 admin admin] PATH_INFO: `/v3/users/9945e4ffbc2548d5962d581aa905bb5f` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:31.517656 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb9c1b1-b1d4-422a-a4a7-40e4cc932681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9945e4ffbc2548d5962d581aa905bb5f)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:31.519656 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb9c1b1-b1d4-422a-a4a7-40e4cc932681 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:31.581610 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-ecb9c1b1-b1d4-422a-a4a7-40e4cc932681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9945e4ffbc2548d5962d581aa905bb5f'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:31.582691 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-ecb9c1b1-b1d4-422a-a4a7-40e4cc932681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9945e4ffbc2548d5962d581aa905bb5f'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:31.591278 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-ecb9c1b1-b1d4-422a-a4a7-40e4cc932681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9945e4ffbc2548d5962d581aa905bb5f'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:31.592113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-ecb9c1b1-b1d4-422a-a4a7-40e4cc932681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9945e4ffbc2548d5962d581aa905bb5f'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:31.608064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-ecb9c1b1-b1d4-422a-a4a7-40e4cc932681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9945e4ffbc2548d5962d581aa905bb5f'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:31.621732 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1175/2369] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:18:31 2022] DELETE /identity/v3/users/9945e4ffbc2548d5962d581aa905bb5f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:18:31.833594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099df3f7-7eb3-43d1-b66f-b018345a5085 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:31.835074 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-099df3f7-7eb3-43d1-b66f-b018345a5085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:31.855390 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099df3f7-7eb3-43d1-b66f-b018345a5085 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:31.931739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afde20f-c797-4f3c-b164-a3253021853a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:31.932548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afde20f-c797-4f3c-b164-a3253021853a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:31.933083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afde20f-c797-4f3c-b164-a3253021853a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:31.933442 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afde20f-c797-4f3c-b164-a3253021853a admin admin] PATH_INFO: `/v3/projects/0cae23fd696d4f579f5c8e6e6d68ce45` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:31.934760 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afde20f-c797-4f3c-b164-a3253021853a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cae23fd696d4f579f5c8e6e6d68ce45)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:31.937075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afde20f-c797-4f3c-b164-a3253021853a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:31.946623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9c1b1-b1d4-422a-a4a7-40e4cc932681 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:31.948511 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ecb9c1b1-b1d4-422a-a4a7-40e4cc932681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:32.034473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-9afde20f-c797-4f3c-b164-a3253021853a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cae23fd696d4f579f5c8e6e6d68ce45', 'request_id': 'req-9afde20f-c797-4f3c-b164-a3253021853a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:32.048580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-9afde20f-c797-4f3c-b164-a3253021853a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cae23fd696d4f579f5c8e6e6d68ce45', 'request_id': 'req-9afde20f-c797-4f3c-b164-a3253021853a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:32.059452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-9afde20f-c797-4f3c-b164-a3253021853a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cae23fd696d4f579f5c8e6e6d68ce45', 'request_id': 'req-9afde20f-c797-4f3c-b164-a3253021853a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:32.073835 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-9afde20f-c797-4f3c-b164-a3253021853a admin admin] The token cache is being invalidate because project 0cae23fd696d4f579f5c8e6e6d68ce45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:18:32.074130 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-9afde20f-c797-4f3c-b164-a3253021853a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:32.075959 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-9afde20f-c797-4f3c-b164-a3253021853a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:32.078741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9c1b1-b1d4-422a-a4a7-40e4cc932681 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:32.082235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1195/2370] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:18:31 2022] DELETE /identity/v3/projects/0cae23fd696d4f579f5c8e6e6d68ce45 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:18:32.083875 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59695e31-7683-46a9-b0d2-fe9a92359cce None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:32.084114 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59695e31-7683-46a9-b0d2-fe9a92359cce None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:32.084393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59695e31-7683-46a9-b0d2-fe9a92359cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:32.122050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-59695e31-7683-46a9-b0d2-fe9a92359cce None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:32.123387 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-59695e31-7683-46a9-b0d2-fe9a92359cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:32.133686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:32.204678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f70db-e531-4abe-bd19-bd02aad1cc1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:32.204868 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:32.205345 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f70db-e531-4abe-bd19-bd02aad1cc1d service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:32.206116 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f70db-e531-4abe-bd19-bd02aad1cc1d service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:32.206416 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f70db-e531-4abe-bd19-bd02aad1cc1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:32.209637 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-7e2f70db-e531-4abe-bd19-bd02aad1cc1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:32.229110 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f70db-e531-4abe-bd19-bd02aad1cc1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:32.229820 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1196/2371] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:18:32.232182 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f70db-e531-4abe-bd19-bd02aad1cc1d service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:32.238836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c51c64-0bc4-487f-b88e-f93b4ff887fc None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:32.239481 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c51c64-0bc4-487f-b88e-f93b4ff887fc None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:32.239481 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c51c64-0bc4-487f-b88e-f93b4ff887fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:32.246631 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1176/2372] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:18:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:18:32.260745 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-47c51c64-0bc4-487f-b88e-f93b4ff887fc None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:32.262317 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-47c51c64-0bc4-487f-b88e-f93b4ff887fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:32.271740 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1197/2373] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:18:32.276452 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f70db-e531-4abe-bd19-bd02aad1cc1d service nova] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:32.289145 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f70db-e531-4abe-bd19-bd02aad1cc1d service nova] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:32.297827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b11a25-58d1-4f89-b081-7b8f09b0230a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:32.298545 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b11a25-58d1-4f89-b081-7b8f09b0230a admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:32.299053 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b11a25-58d1-4f89-b081-7b8f09b0230a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:32.299240 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b11a25-58d1-4f89-b081-7b8f09b0230a admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:32.300104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b11a25-58d1-4f89-b081-7b8f09b0230a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:18:32.300524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b11a25-58d1-4f89-b081-7b8f09b0230a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:32.302237 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b11a25-58d1-4f89-b081-7b8f09b0230a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:32.316146 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1177/2374] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:18:32.320816 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c51c64-0bc4-487f-b88e-f93b4ff887fc None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:32.332160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c51c64-0bc4-487f-b88e-f93b4ff887fc None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:32.338143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c60be-2935-4343-9bd3-3a949a37d7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:32.339282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c60be-2935-4343-9bd3-3a949a37d7aa admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:32.340324 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c60be-2935-4343-9bd3-3a949a37d7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:32.341137 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c60be-2935-4343-9bd3-3a949a37d7aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:32.342730 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c60be-2935-4343-9bd3-3a949a37d7aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:32.347636 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c60be-2935-4343-9bd3-3a949a37d7aa admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:32.405883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1198/2375] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:18:32 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:18:32.413397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b11a25-58d1-4f89-b081-7b8f09b0230a admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:32.433593 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b11a25-58d1-4f89-b081-7b8f09b0230a admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:32.531971 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b25bd-fb01-4daf-b2bc-016993f4cf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:32.532644 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b25bd-fb01-4daf-b2bc-016993f4cf1f admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:32.532924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b25bd-fb01-4daf-b2bc-016993f4cf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:32.533223 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b25bd-fb01-4daf-b2bc-016993f4cf1f admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:32.533866 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b25bd-fb01-4daf-b2bc-016993f4cf1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:32.535908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b25bd-fb01-4daf-b2bc-016993f4cf1f admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:32.561981 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1178/2376] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:18:32 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:18:32.566713 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c60be-2935-4343-9bd3-3a949a37d7aa admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:32.577243 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c60be-2935-4343-9bd3-3a949a37d7aa admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:32.584338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85bcbb-de6e-4090-9570-96196f441301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:32.585094 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85bcbb-de6e-4090-9570-96196f441301 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:32.585435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85bcbb-de6e-4090-9570-96196f441301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:32.586261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85bcbb-de6e-4090-9570-96196f441301 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:32.586733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a85bcbb-de6e-4090-9570-96196f441301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:32.588626 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a85bcbb-de6e-4090-9570-96196f441301 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:32.601704 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1199/2377] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:18:32.607427 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b25bd-fb01-4daf-b2bc-016993f4cf1f admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:32.618742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b25bd-fb01-4daf-b2bc-016993f4cf1f admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:32.627533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a68887-16b5-4135-b03b-576b04e1a4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:32.627771 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a68887-16b5-4135-b03b-576b04e1a4d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:32.630160 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a68887-16b5-4135-b03b-576b04e1a4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:32.630160 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a68887-16b5-4135-b03b-576b04e1a4d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:32.630160 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a68887-16b5-4135-b03b-576b04e1a4d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:32.633283 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a68887-16b5-4135-b03b-576b04e1a4d8 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:32.646176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1179/2378] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:18:32.650237 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85bcbb-de6e-4090-9570-96196f441301 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:32.662114 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85bcbb-de6e-4090-9570-96196f441301 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:32.666874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980cfc6-3ea5-4a94-893a-d78b5f3db996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:32.667555 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980cfc6-3ea5-4a94-893a-d78b5f3db996 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:32.667829 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980cfc6-3ea5-4a94-893a-d78b5f3db996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:32.668101 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980cfc6-3ea5-4a94-893a-d78b5f3db996 admin admin] PATH_INFO: `/v3/projects/0dbf7f1737834b12a093b137169033c5/users/31789e13f7824dae998c99eefc7aaaaf/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:32.686804 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980cfc6-3ea5-4a94-893a-d78b5f3db996 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dbf7f1737834b12a093b137169033c5, user_id=31789e13f7824dae998c99eefc7aaaaf, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:32.689620 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980cfc6-3ea5-4a94-893a-d78b5f3db996 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:32.707455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1200/2379] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:18:32 2022] PUT /identity/v3/projects/0dbf7f1737834b12a093b137169033c5/users/31789e13f7824dae998c99eefc7aaaaf/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:18:32.713677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fac91-d241-4f69-8ca2-866f75340214 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:32.713967 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fac91-d241-4f69-8ca2-866f75340214 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:32.714239 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fac91-d241-4f69-8ca2-866f75340214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:32.737795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-3f2fac91-d241-4f69-8ca2-866f75340214 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:32.739028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-3f2fac91-d241-4f69-8ca2-866f75340214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:32.811262 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1180/2380] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:18:32.822768 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980cfc6-3ea5-4a94-893a-d78b5f3db996 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:32.824709 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-6980cfc6-3ea5-4a94-893a-d78b5f3db996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:32.898930 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980cfc6-3ea5-4a94-893a-d78b5f3db996 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:32.903798 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3813da-0aa7-486a-91a7-9244c8be8a5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:32.904436 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3813da-0aa7-486a-91a7-9244c8be8a5e service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:32.904750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3813da-0aa7-486a-91a7-9244c8be8a5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:32.905022 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3813da-0aa7-486a-91a7-9244c8be8a5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:32.915484 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3813da-0aa7-486a-91a7-9244c8be8a5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:32.916982 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3813da-0aa7-486a-91a7-9244c8be8a5e service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:32.928225 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1201/2381] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:18:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:18:37.890870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96207c-5761-479e-b3bc-02bdd6445102 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:37.891459 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96207c-5761-479e-b3bc-02bdd6445102 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:37.891459 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96207c-5761-479e-b3bc-02bdd6445102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:37.927495 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-bd96207c-5761-479e-b3bc-02bdd6445102 None None] MFA Rules not processed for user `31789e13f7824dae998c99eefc7aaaaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:37.928784 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-bd96207c-5761-479e-b3bc-02bdd6445102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:37.939129 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:38.004970 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1181/2382] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:18:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:38.009718 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3813da-0aa7-486a-91a7-9244c8be8a5e service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:38.021384 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3813da-0aa7-486a-91a7-9244c8be8a5e service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:38.026219 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150d037-d31a-43a3-b906-465fa2155cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:38.026842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150d037-d31a-43a3-b906-465fa2155cac admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:38.027119 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150d037-d31a-43a3-b906-465fa2155cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:38.027390 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150d037-d31a-43a3-b906-465fa2155cac admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:38.028072 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150d037-d31a-43a3-b906-465fa2155cac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:38.030026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150d037-d31a-43a3-b906-465fa2155cac admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:38.059080 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1202/2383] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:18:38 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:18:38.063458 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96207c-5761-479e-b3bc-02bdd6445102 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:38.075197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96207c-5761-479e-b3bc-02bdd6445102 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:38.137838 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b2890-53d1-4330-83fe-319dc527cb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:38.138454 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b2890-53d1-4330-83fe-319dc527cb41 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:38.138732 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b2890-53d1-4330-83fe-319dc527cb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:38.139000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b2890-53d1-4330-83fe-319dc527cb41 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:38.139661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b2890-53d1-4330-83fe-319dc527cb41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:38.142401 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b2890-53d1-4330-83fe-319dc527cb41 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:38.172616 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1182/2384] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:18:38 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:18:38.176251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150d037-d31a-43a3-b906-465fa2155cac admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:38.192517 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150d037-d31a-43a3-b906-465fa2155cac admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:38.198422 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55add5c3-f281-402c-acb4-433a8c3ca001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:38.199461 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55add5c3-f281-402c-acb4-433a8c3ca001 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:38.199876 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55add5c3-f281-402c-acb4-433a8c3ca001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:38.200279 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55add5c3-f281-402c-acb4-433a8c3ca001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:38.201296 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55add5c3-f281-402c-acb4-433a8c3ca001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:38.203976 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55add5c3-f281-402c-acb4-433a8c3ca001 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:38.216785 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1203/2385] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:18:38.221277 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b2890-53d1-4330-83fe-319dc527cb41 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:38.232855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b2890-53d1-4330-83fe-319dc527cb41 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:38.237912 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d75df-fa55-4ea9-b1a4-e85fa62d3fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:38.238584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d75df-fa55-4ea9-b1a4-e85fa62d3fef admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:38.238864 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d75df-fa55-4ea9-b1a4-e85fa62d3fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:38.239138 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d75df-fa55-4ea9-b1a4-e85fa62d3fef admin admin] PATH_INFO: `/v3/projects/a4067667ba2e4051bc27bb45bd7b5bad/users/8c976d7293bc495a82ce33404c638103/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:38.256785 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d75df-fa55-4ea9-b1a4-e85fa62d3fef admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4067667ba2e4051bc27bb45bd7b5bad, user_id=8c976d7293bc495a82ce33404c638103, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:38.260928 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d75df-fa55-4ea9-b1a4-e85fa62d3fef admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:38.283298 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1183/2386] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:18:38 2022] PUT /identity/v3/projects/a4067667ba2e4051bc27bb45bd7b5bad/users/8c976d7293bc495a82ce33404c638103/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:18:38.291183 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55add5c3-f281-402c-acb4-433a8c3ca001 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:38.301867 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55add5c3-f281-402c-acb4-433a8c3ca001 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:38.374249 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f998ff-a934-4ad0-89dd-c43830b5d8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:38.375217 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f998ff-a934-4ad0-89dd-c43830b5d8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:38.375502 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f998ff-a934-4ad0-89dd-c43830b5d8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:38.375777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f998ff-a934-4ad0-89dd-c43830b5d8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:38.376514 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f998ff-a934-4ad0-89dd-c43830b5d8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:38.378328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f998ff-a934-4ad0-89dd-c43830b5d8e7 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:38.380585 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:38.400438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d75df-fa55-4ea9-b1a4-e85fa62d3fef admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:38.400729 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1204/2387] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:18:38.415100 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d75df-fa55-4ea9-b1a4-e85fa62d3fef admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:38.421606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f246a-b0be-4411-aa62-6b24201dd197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:38.422349 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f246a-b0be-4411-aa62-6b24201dd197 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:38.422630 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f246a-b0be-4411-aa62-6b24201dd197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:38.422913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f246a-b0be-4411-aa62-6b24201dd197 admin admin] PATH_INFO: `/v3/projects/a4067667ba2e4051bc27bb45bd7b5bad/users/8c976d7293bc495a82ce33404c638103/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:38.425013 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f246a-b0be-4411-aa62-6b24201dd197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4067667ba2e4051bc27bb45bd7b5bad, user_id=8c976d7293bc495a82ce33404c638103, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:38.427711 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f246a-b0be-4411-aa62-6b24201dd197 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:38.442291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1184/2388] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:18:38 2022] PUT /identity/v3/projects/a4067667ba2e4051bc27bb45bd7b5bad/users/8c976d7293bc495a82ce33404c638103/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:18:44.150507 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019707bb-4323-4df1-8e87-b83c4908eed6 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:44.151161 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019707bb-4323-4df1-8e87-b83c4908eed6 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:44.151161 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019707bb-4323-4df1-8e87-b83c4908eed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:44.182522 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-019707bb-4323-4df1-8e87-b83c4908eed6 None None] MFA Rules not processed for user `8c976d7293bc495a82ce33404c638103`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:44.183803 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-019707bb-4323-4df1-8e87-b83c4908eed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:44.270224 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1205/2389] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:18:44 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:44.285037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f246a-b0be-4411-aa62-6b24201dd197 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:44.286939 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-574f246a-b0be-4411-aa62-6b24201dd197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:44.349684 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:44.366527 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f246a-b0be-4411-aa62-6b24201dd197 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:44.372178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785c4fc-58f0-4af7-8ece-e52ac5b8eeff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:44.372953 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785c4fc-58f0-4af7-8ece-e52ac5b8eeff service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:44.373229 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785c4fc-58f0-4af7-8ece-e52ac5b8eeff service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:44.373495 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785c4fc-58f0-4af7-8ece-e52ac5b8eeff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:44.432582 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785c4fc-58f0-4af7-8ece-e52ac5b8eeff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:44.435626 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785c4fc-58f0-4af7-8ece-e52ac5b8eeff service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:44.455484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1185/2390] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:18:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:18:45.845055 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25034973-026e-4357-bebc-9d3e8c05e9bb None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:45.845790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25034973-026e-4357-bebc-9d3e8c05e9bb None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:45.845790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25034973-026e-4357-bebc-9d3e8c05e9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:45.868674 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-25034973-026e-4357-bebc-9d3e8c05e9bb None None] MFA Rules not processed for user `31789e13f7824dae998c99eefc7aaaaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:45.870180 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-25034973-026e-4357-bebc-9d3e8c05e9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:45.884596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1206/2391] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:18:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:46.454854 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785c4fc-58f0-4af7-8ece-e52ac5b8eeff service nova] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:46.547925 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785c4fc-58f0-4af7-8ece-e52ac5b8eeff service nova] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:46.552443 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8317de-6fdf-4115-b0cb-b5d0e6cb4b63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:46.553459 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8317de-6fdf-4115-b0cb-b5d0e6cb4b63 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:46.553714 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8317de-6fdf-4115-b0cb-b5d0e6cb4b63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:46.553992 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8317de-6fdf-4115-b0cb-b5d0e6cb4b63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:46.556134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-1d8317de-6fdf-4115-b0cb-b5d0e6cb4b63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:46.636711 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8317de-6fdf-4115-b0cb-b5d0e6cb4b63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:46.638343 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8317de-6fdf-4115-b0cb-b5d0e6cb4b63 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:46.655424 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1186/2392] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:18:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:18:50.488097 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25034973-026e-4357-bebc-9d3e8c05e9bb None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:50.490513 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-25034973-026e-4357-bebc-9d3e8c05e9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:50.512198 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25034973-026e-4357-bebc-9d3e8c05e9bb None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:50.519904 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e094dd8-7ba3-43fc-9926-fa5826b80ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:50.520804 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e094dd8-7ba3-43fc-9926-fa5826b80ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:50.520939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e094dd8-7ba3-43fc-9926-fa5826b80ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:50.521190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e094dd8-7ba3-43fc-9926-fa5826b80ac3 admin admin] PATH_INFO: `/v3/users/3af3a5c1c2f94e51bee936b4762dc14c` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:50.522448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e094dd8-7ba3-43fc-9926-fa5826b80ac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3af3a5c1c2f94e51bee936b4762dc14c)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:50.525313 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e094dd8-7ba3-43fc-9926-fa5826b80ac3 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:50.593441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-9e094dd8-7ba3-43fc-9926-fa5826b80ac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3af3a5c1c2f94e51bee936b4762dc14c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:50.604378 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-9e094dd8-7ba3-43fc-9926-fa5826b80ac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3af3a5c1c2f94e51bee936b4762dc14c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:50.612562 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-9e094dd8-7ba3-43fc-9926-fa5826b80ac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3af3a5c1c2f94e51bee936b4762dc14c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:50.613238 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-9e094dd8-7ba3-43fc-9926-fa5826b80ac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3af3a5c1c2f94e51bee936b4762dc14c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:50.613851 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-9e094dd8-7ba3-43fc-9926-fa5826b80ac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3af3a5c1c2f94e51bee936b4762dc14c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:50.633142 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1207/2393] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:18:50 2022] DELETE /identity/v3/users/3af3a5c1c2f94e51bee936b4762dc14c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:18:50.818284 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8317de-6fdf-4115-b0cb-b5d0e6cb4b63 service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:50.835590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8317de-6fdf-4115-b0cb-b5d0e6cb4b63 service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:50.885893 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:50.899032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d759aba-be96-4bcd-aab6-22ec4ac167ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:50.900048 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d759aba-be96-4bcd-aab6-22ec4ac167ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:50.900399 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d759aba-be96-4bcd-aab6-22ec4ac167ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:50.900770 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d759aba-be96-4bcd-aab6-22ec4ac167ee admin admin] PATH_INFO: `/v3/projects/29b33a1fb52146918d3c0bd0d23d2dfa` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:50.902231 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d759aba-be96-4bcd-aab6-22ec4ac167ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=29b33a1fb52146918d3c0bd0d23d2dfa)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:50.904431 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d759aba-be96-4bcd-aab6-22ec4ac167ee admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:50.954338 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3d759aba-be96-4bcd-aab6-22ec4ac167ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29b33a1fb52146918d3c0bd0d23d2dfa', 'request_id': 'req-3d759aba-be96-4bcd-aab6-22ec4ac167ee'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:50.961336 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3d759aba-be96-4bcd-aab6-22ec4ac167ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29b33a1fb52146918d3c0bd0d23d2dfa', 'request_id': 'req-3d759aba-be96-4bcd-aab6-22ec4ac167ee'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:50.967063 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3d759aba-be96-4bcd-aab6-22ec4ac167ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29b33a1fb52146918d3c0bd0d23d2dfa', 'request_id': 'req-3d759aba-be96-4bcd-aab6-22ec4ac167ee'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:50.980759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3d759aba-be96-4bcd-aab6-22ec4ac167ee admin admin] The token cache is being invalidate because project 29b33a1fb52146918d3c0bd0d23d2dfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:18:50.981014 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3d759aba-be96-4bcd-aab6-22ec4ac167ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:50.981643 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3d759aba-be96-4bcd-aab6-22ec4ac167ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:50.983167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1187/2394] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:18:50 2022] DELETE /identity/v3/projects/29b33a1fb52146918d3c0bd0d23d2dfa => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:18:50.989216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283ccca-0152-439c-b00d-d0f10deb28c3 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:50.989558 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283ccca-0152-439c-b00d-d0f10deb28c3 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:50.989839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283ccca-0152-439c-b00d-d0f10deb28c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:51.009493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-5283ccca-0152-439c-b00d-d0f10deb28c3 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:51.010713 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5283ccca-0152-439c-b00d-d0f10deb28c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:51.058518 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:51.089557 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1208/2395] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:51.091441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06705957-5e0f-4fe5-b570-601b3727031b None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:51.091650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06705957-5e0f-4fe5-b570-601b3727031b None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:51.091931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06705957-5e0f-4fe5-b570-601b3727031b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:51.115886 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-06705957-5e0f-4fe5-b570-601b3727031b None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:51.117143 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-06705957-5e0f-4fe5-b570-601b3727031b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:51.125152 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1188/2396] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:18:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:18:51.129723 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283ccca-0152-439c-b00d-d0f10deb28c3 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:51.141041 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283ccca-0152-439c-b00d-d0f10deb28c3 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:51.146214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b480658-1d72-444b-849e-4466ddbb1e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:51.146881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b480658-1d72-444b-849e-4466ddbb1e1d admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:51.147104 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b480658-1d72-444b-849e-4466ddbb1e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:51.147375 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b480658-1d72-444b-849e-4466ddbb1e1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:51.148988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b480658-1d72-444b-849e-4466ddbb1e1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:18:51.149309 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b480658-1d72-444b-849e-4466ddbb1e1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:51.151321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b480658-1d72-444b-849e-4466ddbb1e1d admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:51.164508 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1209/2397] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:18:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:18:51.169169 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06705957-5e0f-4fe5-b570-601b3727031b None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:51.179804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06705957-5e0f-4fe5-b570-601b3727031b None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:51.185002 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dca3c4-0513-41ae-ba47-1239ac8ff082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:51.185642 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dca3c4-0513-41ae-ba47-1239ac8ff082 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:51.185960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dca3c4-0513-41ae-ba47-1239ac8ff082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:51.186247 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dca3c4-0513-41ae-ba47-1239ac8ff082 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:51.187036 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dca3c4-0513-41ae-ba47-1239ac8ff082 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:51.189225 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dca3c4-0513-41ae-ba47-1239ac8ff082 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:51.225617 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1189/2398] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:18:51 2022] POST /identity/v3/projects => generated 355 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:18:51.232437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b480658-1d72-444b-849e-4466ddbb1e1d admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:51.250187 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b480658-1d72-444b-849e-4466ddbb1e1d admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:51.325794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09e622-d2a6-4ebf-84dd-bebf90935ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:51.326551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09e622-d2a6-4ebf-84dd-bebf90935ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:51.326927 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09e622-d2a6-4ebf-84dd-bebf90935ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:51.327812 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09e622-d2a6-4ebf-84dd-bebf90935ae6 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:51.328733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09e622-d2a6-4ebf-84dd-bebf90935ae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:51.331318 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09e622-d2a6-4ebf-84dd-bebf90935ae6 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:51.360033 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1210/2399] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:18:51 2022] POST /identity/v3/users => generated 278 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:18:51.361473 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dca3c4-0513-41ae-ba47-1239ac8ff082 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:51.372287 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dca3c4-0513-41ae-ba47-1239ac8ff082 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:51.377230 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43653f49-2a13-4eb8-aea4-d1104d07e299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:51.377952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43653f49-2a13-4eb8-aea4-d1104d07e299 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:51.378136 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43653f49-2a13-4eb8-aea4-d1104d07e299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:51.378412 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43653f49-2a13-4eb8-aea4-d1104d07e299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:51.379117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43653f49-2a13-4eb8-aea4-d1104d07e299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:51.380780 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43653f49-2a13-4eb8-aea4-d1104d07e299 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:51.392034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1190/2400] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:18:51.396418 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09e622-d2a6-4ebf-84dd-bebf90935ae6 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:51.407735 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09e622-d2a6-4ebf-84dd-bebf90935ae6 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:51.412625 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920e0ad-e4c1-40a1-a7be-5a89054ceb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:51.413365 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920e0ad-e4c1-40a1-a7be-5a89054ceb74 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:51.413648 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920e0ad-e4c1-40a1-a7be-5a89054ceb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:51.413926 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920e0ad-e4c1-40a1-a7be-5a89054ceb74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:51.414624 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920e0ad-e4c1-40a1-a7be-5a89054ceb74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:51.416242 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920e0ad-e4c1-40a1-a7be-5a89054ceb74 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:51.433083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1211/2401] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:18:51.434442 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43653f49-2a13-4eb8-aea4-d1104d07e299 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:51.445699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43653f49-2a13-4eb8-aea4-d1104d07e299 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:51.451871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424e461-78f5-49f2-bd7e-c0f2f85fea40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:51.453202 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424e461-78f5-49f2-bd7e-c0f2f85fea40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:51.453555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424e461-78f5-49f2-bd7e-c0f2f85fea40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:51.454034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424e461-78f5-49f2-bd7e-c0f2f85fea40 admin admin] PATH_INFO: `/v3/projects/d5860fa05caf408fa8b88d3b6ed51778/users/0a99e50a09b244adbaf8204c81212978/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:51.473060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424e461-78f5-49f2-bd7e-c0f2f85fea40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5860fa05caf408fa8b88d3b6ed51778, user_id=0a99e50a09b244adbaf8204c81212978, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:51.475761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424e461-78f5-49f2-bd7e-c0f2f85fea40 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:51.490466 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1191/2402] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:18:51 2022] PUT /identity/v3/projects/d5860fa05caf408fa8b88d3b6ed51778/users/0a99e50a09b244adbaf8204c81212978/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:18:51.501175 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c786106-b5a2-42f0-a3c3-c8b9093b87b8 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:51.501664 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c786106-b5a2-42f0-a3c3-c8b9093b87b8 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:51.501760 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c786106-b5a2-42f0-a3c3-c8b9093b87b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:51.533575 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-8c786106-b5a2-42f0-a3c3-c8b9093b87b8 None None] MFA Rules not processed for user `0a99e50a09b244adbaf8204c81212978`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:51.534819 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-8c786106-b5a2-42f0-a3c3-c8b9093b87b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:51.612002 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1212/2403] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:18:51 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:51.616556 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424e461-78f5-49f2-bd7e-c0f2f85fea40 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:51.683677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424e461-78f5-49f2-bd7e-c0f2f85fea40 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:51.689105 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218532f4-fb0a-415f-8a5a-95b60494939c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:51.689719 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218532f4-fb0a-415f-8a5a-95b60494939c admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:51.689998 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218532f4-fb0a-415f-8a5a-95b60494939c admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:51.690270 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218532f4-fb0a-415f-8a5a-95b60494939c admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:51.690930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218532f4-fb0a-415f-8a5a-95b60494939c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:51.692879 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218532f4-fb0a-415f-8a5a-95b60494939c admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:51.719778 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:51.731130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1192/2404] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:18:51 2022] POST /identity/v3/projects => generated 355 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:18:51.735410 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c786106-b5a2-42f0-a3c3-c8b9093b87b8 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:51.755323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c786106-b5a2-42f0-a3c3-c8b9093b87b8 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:51.805563 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:51.823953 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7c65f-c722-4090-a01b-6921a8d3b171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:51.824599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7c65f-c722-4090-a01b-6921a8d3b171 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:51.824959 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7c65f-c722-4090-a01b-6921a8d3b171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:51.825145 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7c65f-c722-4090-a01b-6921a8d3b171 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:51.825805 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7c65f-c722-4090-a01b-6921a8d3b171 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:51.828336 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7c65f-c722-4090-a01b-6921a8d3b171 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:51.854951 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1213/2405] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:18:51 2022] POST /identity/v3/users => generated 284 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:18:51.863144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218532f4-fb0a-415f-8a5a-95b60494939c admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:51.879558 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218532f4-fb0a-415f-8a5a-95b60494939c admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:51.884726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d2297-23d0-41eb-b0bb-ceeec7da4225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:51.885352 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d2297-23d0-41eb-b0bb-ceeec7da4225 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:51.885630 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d2297-23d0-41eb-b0bb-ceeec7da4225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:51.885901 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d2297-23d0-41eb-b0bb-ceeec7da4225 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:51.886600 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d2297-23d0-41eb-b0bb-ceeec7da4225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:51.888245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d2297-23d0-41eb-b0bb-ceeec7da4225 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:51.900811 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1193/2406] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:18:51.905945 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7c65f-c722-4090-a01b-6921a8d3b171 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:51.916707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7c65f-c722-4090-a01b-6921a8d3b171 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:51.921799 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a796bd-c18d-48e1-9562-67a62750352d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:51.922529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a796bd-c18d-48e1-9562-67a62750352d admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:51.922808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a796bd-c18d-48e1-9562-67a62750352d admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:51.923081 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a796bd-c18d-48e1-9562-67a62750352d admin admin] PATH_INFO: `/v3/projects/8bafeef4662f474891ed88e9f4b8c359/users/1a6cc5ac44bd46e2bcbef6aad2443733/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:51.939438 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a796bd-c18d-48e1-9562-67a62750352d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bafeef4662f474891ed88e9f4b8c359, user_id=1a6cc5ac44bd46e2bcbef6aad2443733, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:51.942281 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a796bd-c18d-48e1-9562-67a62750352d admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:51.956971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1214/2407] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:18:51 2022] PUT /identity/v3/projects/8bafeef4662f474891ed88e9f4b8c359/users/1a6cc5ac44bd46e2bcbef6aad2443733/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:18:51.960965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d2297-23d0-41eb-b0bb-ceeec7da4225 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:51.971693 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d2297-23d0-41eb-b0bb-ceeec7da4225 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:52.029825 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977b192-857d-4df8-8e2a-59586c31ecb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:52.030915 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977b192-857d-4df8-8e2a-59586c31ecb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:52.031378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977b192-857d-4df8-8e2a-59586c31ecb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:52.031910 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977b192-857d-4df8-8e2a-59586c31ecb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:52.033191 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977b192-857d-4df8-8e2a-59586c31ecb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:52.035423 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977b192-857d-4df8-8e2a-59586c31ecb3 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:52.049072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1194/2408] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:18:52.053998 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a796bd-c18d-48e1-9562-67a62750352d admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:52.065194 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a796bd-c18d-48e1-9562-67a62750352d admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:52.069830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a696d-3843-4c68-a06c-9ea2c6d0833b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:52.070497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a696d-3843-4c68-a06c-9ea2c6d0833b admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:52.070806 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a696d-3843-4c68-a06c-9ea2c6d0833b admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:52.071085 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a696d-3843-4c68-a06c-9ea2c6d0833b admin admin] PATH_INFO: `/v3/projects/8bafeef4662f474891ed88e9f4b8c359/users/1a6cc5ac44bd46e2bcbef6aad2443733/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:52.073079 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a696d-3843-4c68-a06c-9ea2c6d0833b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bafeef4662f474891ed88e9f4b8c359, user_id=1a6cc5ac44bd46e2bcbef6aad2443733, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:52.075741 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a696d-3843-4c68-a06c-9ea2c6d0833b admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:52.086143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1215/2409] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:18:52 2022] PUT /identity/v3/projects/8bafeef4662f474891ed88e9f4b8c359/users/1a6cc5ac44bd46e2bcbef6aad2443733/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:18:52.097984 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5dbee-380d-4755-82c9-43418efb83e3 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:52.098298 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5dbee-380d-4755-82c9-43418efb83e3 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:52.098651 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5dbee-380d-4755-82c9-43418efb83e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:52.138089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-ffe5dbee-380d-4755-82c9-43418efb83e3 None None] MFA Rules not processed for user `1a6cc5ac44bd46e2bcbef6aad2443733`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:52.139239 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ffe5dbee-380d-4755-82c9-43418efb83e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:52.214107 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1195/2410] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:18:52 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:52.225888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a696d-3843-4c68-a06c-9ea2c6d0833b admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:52.227622 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a36a696d-3843-4c68-a06c-9ea2c6d0833b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:52.294607 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a696d-3843-4c68-a06c-9ea2c6d0833b admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:52.301600 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af243a0-4cc8-47c9-8c85-449631a79e80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:52.302668 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af243a0-4cc8-47c9-8c85-449631a79e80 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:52.303169 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af243a0-4cc8-47c9-8c85-449631a79e80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:52.303670 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af243a0-4cc8-47c9-8c85-449631a79e80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:52.318121 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af243a0-4cc8-47c9-8c85-449631a79e80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:52.319579 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af243a0-4cc8-47c9-8c85-449631a79e80 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:52.329134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1216/2411] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:18:52 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:18:52.405091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5dbee-380d-4755-82c9-43418efb83e3 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:52.415374 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5dbee-380d-4755-82c9-43418efb83e3 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:52.418992 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb90f23-f9c0-4f3c-a1f0-e9b07d7c3a0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:52.419599 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb90f23-f9c0-4f3c-a1f0-e9b07d7c3a0b service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:52.419878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb90f23-f9c0-4f3c-a1f0-e9b07d7c3a0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:52.420149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb90f23-f9c0-4f3c-a1f0-e9b07d7c3a0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:52.607822 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb90f23-f9c0-4f3c-a1f0-e9b07d7c3a0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:52.609966 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb90f23-f9c0-4f3c-a1f0-e9b07d7c3a0b service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:52.618780 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1196/2412] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:18:52 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:02.324880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=51,keystone:INSERT=5 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:02.614791 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=96 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:03.878390 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af243a0-4cc8-47c9-8c85-449631a79e80 service cinder] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:03.879936 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3af243a0-4cc8-47c9-8c85-449631a79e80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:03.953456 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af243a0-4cc8-47c9-8c85-449631a79e80 service cinder] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:03.957729 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028279cb-7814-4f33-aff7-2ece1071a1c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:03.958381 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028279cb-7814-4f33-aff7-2ece1071a1c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:03.958658 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028279cb-7814-4f33-aff7-2ece1071a1c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:03.958931 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028279cb-7814-4f33-aff7-2ece1071a1c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:03.960866 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-028279cb-7814-4f33-aff7-2ece1071a1c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:03.974933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028279cb-7814-4f33-aff7-2ece1071a1c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:03.976588 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028279cb-7814-4f33-aff7-2ece1071a1c4 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:03.991212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1217/2413] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:04.613916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb90f23-f9c0-4f3c-a1f0-e9b07d7c3a0b service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:04.616146 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-5cb90f23-f9c0-4f3c-a1f0-e9b07d7c3a0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:04.699945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb90f23-f9c0-4f3c-a1f0-e9b07d7c3a0b service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:04.705500 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27b638-24ec-4727-b01a-3bcc17567c7e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07e3e25e6a8466fb237dcb836aecda2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:04.706131 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27b638-24ec-4727-b01a-3bcc17567c7e service placement] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:04.706450 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27b638-24ec-4727-b01a-3bcc17567c7e service placement] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:04.706804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27b638-24ec-4727-b01a-3bcc17567c7e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:04.708685 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ff27b638-24ec-4727-b01a-3bcc17567c7e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:04.721516 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff27b638-24ec-4727-b01a-3bcc17567c7e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:04.723123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff27b638-24ec-4727-b01a-3bcc17567c7e service placement] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:04.735388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1197/2414] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:08.677372 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028279cb-7814-4f33-aff7-2ece1071a1c4 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:08.778605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028279cb-7814-4f33-aff7-2ece1071a1c4 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:08.786194 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434dc20-17a0-47db-b7d3-4c372f9619c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:08.788645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434dc20-17a0-47db-b7d3-4c372f9619c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:08.788889 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434dc20-17a0-47db-b7d3-4c372f9619c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:08.789293 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434dc20-17a0-47db-b7d3-4c372f9619c4 admin admin] PATH_INFO: `/v3/users/0a99e50a09b244adbaf8204c81212978` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:08.793016 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8434dc20-17a0-47db-b7d3-4c372f9619c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a99e50a09b244adbaf8204c81212978)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:08.795666 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8434dc20-17a0-47db-b7d3-4c372f9619c4 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:08.862053 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-8434dc20-17a0-47db-b7d3-4c372f9619c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a99e50a09b244adbaf8204c81212978'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:08.870365 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-8434dc20-17a0-47db-b7d3-4c372f9619c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a99e50a09b244adbaf8204c81212978'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:08.879900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-8434dc20-17a0-47db-b7d3-4c372f9619c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a99e50a09b244adbaf8204c81212978'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:08.880678 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-8434dc20-17a0-47db-b7d3-4c372f9619c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a99e50a09b244adbaf8204c81212978'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:08.881660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-8434dc20-17a0-47db-b7d3-4c372f9619c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a99e50a09b244adbaf8204c81212978'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:08.909827 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1218/2415] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:08 2022] DELETE /identity/v3/users/0a99e50a09b244adbaf8204c81212978 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:08.915275 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367384f8-88eb-4a9b-bd0a-b4ba42ee51bc None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:08.915961 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367384f8-88eb-4a9b-bd0a-b4ba42ee51bc None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:08.915961 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367384f8-88eb-4a9b-bd0a-b4ba42ee51bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:08.938123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-367384f8-88eb-4a9b-bd0a-b4ba42ee51bc None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:08.939352 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-367384f8-88eb-4a9b-bd0a-b4ba42ee51bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:09.022228 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1198/2416] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:09.032806 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434dc20-17a0-47db-b7d3-4c372f9619c4 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:09.034302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-8434dc20-17a0-47db-b7d3-4c372f9619c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:09.064877 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:09.101653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434dc20-17a0-47db-b7d3-4c372f9619c4 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:09.105773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcfe33-d5c7-48db-afc4-5da975dfe7e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:09.106408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcfe33-d5c7-48db-afc4-5da975dfe7e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:09.106693 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcfe33-d5c7-48db-afc4-5da975dfe7e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:09.106979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcfe33-d5c7-48db-afc4-5da975dfe7e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:09.118420 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbcfe33-d5c7-48db-afc4-5da975dfe7e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:09.120251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbcfe33-d5c7-48db-afc4-5da975dfe7e7 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:09.135449 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1219/2417] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:19:09.642772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367384f8-88eb-4a9b-bd0a-b4ba42ee51bc None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:09.645621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-367384f8-88eb-4a9b-bd0a-b4ba42ee51bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:09.670198 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367384f8-88eb-4a9b-bd0a-b4ba42ee51bc None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:09.675807 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686131db-a5a2-4983-8d9a-8932ae082d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:09.676307 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686131db-a5a2-4983-8d9a-8932ae082d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:09.676654 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686131db-a5a2-4983-8d9a-8932ae082d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:09.676930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686131db-a5a2-4983-8d9a-8932ae082d6f admin admin] PATH_INFO: `/v3/projects/d5860fa05caf408fa8b88d3b6ed51778` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:09.678938 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686131db-a5a2-4983-8d9a-8932ae082d6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5860fa05caf408fa8b88d3b6ed51778)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:09.680348 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686131db-a5a2-4983-8d9a-8932ae082d6f admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:09.738723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-686131db-a5a2-4983-8d9a-8932ae082d6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5860fa05caf408fa8b88d3b6ed51778', 'request_id': 'req-686131db-a5a2-4983-8d9a-8932ae082d6f'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:09.745897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-686131db-a5a2-4983-8d9a-8932ae082d6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5860fa05caf408fa8b88d3b6ed51778', 'request_id': 'req-686131db-a5a2-4983-8d9a-8932ae082d6f'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:09.753151 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-686131db-a5a2-4983-8d9a-8932ae082d6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5860fa05caf408fa8b88d3b6ed51778', 'request_id': 'req-686131db-a5a2-4983-8d9a-8932ae082d6f'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:09.775200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-686131db-a5a2-4983-8d9a-8932ae082d6f admin admin] The token cache is being invalidate because project d5860fa05caf408fa8b88d3b6ed51778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:09.775744 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-686131db-a5a2-4983-8d9a-8932ae082d6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:09.776940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-686131db-a5a2-4983-8d9a-8932ae082d6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:09.783378 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1199/2418] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:09 2022] DELETE /identity/v3/projects/d5860fa05caf408fa8b88d3b6ed51778 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:09.785278 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcfe33-d5c7-48db-afc4-5da975dfe7e7 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:09.800121 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcfe33-d5c7-48db-afc4-5da975dfe7e7 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:09.893672 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565df95f-332a-45ea-b153-1740ffd051b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:09.894253 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565df95f-332a-45ea-b153-1740ffd051b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:09.894530 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565df95f-332a-45ea-b153-1740ffd051b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:09.894809 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565df95f-332a-45ea-b153-1740ffd051b3 admin admin] PATH_INFO: `/v3/users/1a6cc5ac44bd46e2bcbef6aad2443733` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:09.896113 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565df95f-332a-45ea-b153-1740ffd051b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a6cc5ac44bd46e2bcbef6aad2443733)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:09.898154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565df95f-332a-45ea-b153-1740ffd051b3 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:09.962504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-565df95f-332a-45ea-b153-1740ffd051b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a6cc5ac44bd46e2bcbef6aad2443733'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:09.975850 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-565df95f-332a-45ea-b153-1740ffd051b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a6cc5ac44bd46e2bcbef6aad2443733'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:09.990554 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-565df95f-332a-45ea-b153-1740ffd051b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a6cc5ac44bd46e2bcbef6aad2443733'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:09.991475 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-565df95f-332a-45ea-b153-1740ffd051b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a6cc5ac44bd46e2bcbef6aad2443733'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:09.992215 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-565df95f-332a-45ea-b153-1740ffd051b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a6cc5ac44bd46e2bcbef6aad2443733'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:10.018773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1220/2419] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:09 2022] DELETE /identity/v3/users/1a6cc5ac44bd46e2bcbef6aad2443733 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:10.362488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686131db-a5a2-4983-8d9a-8932ae082d6f admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:10.364732 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-686131db-a5a2-4983-8d9a-8932ae082d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:10.384995 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686131db-a5a2-4983-8d9a-8932ae082d6f admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:10.398706 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:10.473963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf8ba5-8bfd-4fa3-b6b7-453c66dc5e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:10.474861 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf8ba5-8bfd-4fa3-b6b7-453c66dc5e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:10.481524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf8ba5-8bfd-4fa3-b6b7-453c66dc5e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:10.481524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf8ba5-8bfd-4fa3-b6b7-453c66dc5e46 admin admin] PATH_INFO: `/v3/projects/8bafeef4662f474891ed88e9f4b8c359` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:10.481524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf8ba5-8bfd-4fa3-b6b7-453c66dc5e46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bafeef4662f474891ed88e9f4b8c359)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:10.481524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf8ba5-8bfd-4fa3-b6b7-453c66dc5e46 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:10.558284 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-03cf8ba5-8bfd-4fa3-b6b7-453c66dc5e46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bafeef4662f474891ed88e9f4b8c359', 'request_id': 'req-03cf8ba5-8bfd-4fa3-b6b7-453c66dc5e46'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:10.565264 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-03cf8ba5-8bfd-4fa3-b6b7-453c66dc5e46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bafeef4662f474891ed88e9f4b8c359', 'request_id': 'req-03cf8ba5-8bfd-4fa3-b6b7-453c66dc5e46'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:10.573000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-03cf8ba5-8bfd-4fa3-b6b7-453c66dc5e46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bafeef4662f474891ed88e9f4b8c359', 'request_id': 'req-03cf8ba5-8bfd-4fa3-b6b7-453c66dc5e46'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:10.590754 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-03cf8ba5-8bfd-4fa3-b6b7-453c66dc5e46 admin admin] The token cache is being invalidate because project 8bafeef4662f474891ed88e9f4b8c359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:10.591134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-03cf8ba5-8bfd-4fa3-b6b7-453c66dc5e46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:10.592007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-03cf8ba5-8bfd-4fa3-b6b7-453c66dc5e46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:10.594348 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1200/2420] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:10 2022] DELETE /identity/v3/projects/8bafeef4662f474891ed88e9f4b8c359 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:10.600892 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80604616-3912-42d8-b5d6-bd9abb590b19 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:10.601330 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80604616-3912-42d8-b5d6-bd9abb590b19 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:10.601540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80604616-3912-42d8-b5d6-bd9abb590b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:10.625536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-80604616-3912-42d8-b5d6-bd9abb590b19 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:10.626770 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-80604616-3912-42d8-b5d6-bd9abb590b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:10.674686 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:10.697797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1221/2421] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:10.706806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e156ca-1f57-4dc2-9c0f-3c7f98d67b64 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:10.707110 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e156ca-1f57-4dc2-9c0f-3c7f98d67b64 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:10.707464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e156ca-1f57-4dc2-9c0f-3c7f98d67b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:10.727522 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-92e156ca-1f57-4dc2-9c0f-3c7f98d67b64 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:10.728966 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-92e156ca-1f57-4dc2-9c0f-3c7f98d67b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:10.736922 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1201/2422] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:19:10.741434 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80604616-3912-42d8-b5d6-bd9abb590b19 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:10.751441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80604616-3912-42d8-b5d6-bd9abb590b19 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:10.756177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716ff38-1dc4-4496-beae-8fe859f34792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:10.756834 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716ff38-1dc4-4496-beae-8fe859f34792 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:10.757179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716ff38-1dc4-4496-beae-8fe859f34792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:10.757463 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716ff38-1dc4-4496-beae-8fe859f34792 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:10.758168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716ff38-1dc4-4496-beae-8fe859f34792 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:10.758572 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716ff38-1dc4-4496-beae-8fe859f34792 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:10.760101 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716ff38-1dc4-4496-beae-8fe859f34792 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:10.773443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1222/2423] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:10.784315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e156ca-1f57-4dc2-9c0f-3c7f98d67b64 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:10.796957 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e156ca-1f57-4dc2-9c0f-3c7f98d67b64 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:10.804853 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc04ea8-08ff-495a-807c-8d066410e62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:10.805959 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc04ea8-08ff-495a-807c-8d066410e62d admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:10.806444 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc04ea8-08ff-495a-807c-8d066410e62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:10.806966 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc04ea8-08ff-495a-807c-8d066410e62d admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:10.808163 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc04ea8-08ff-495a-807c-8d066410e62d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:10.812517 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc04ea8-08ff-495a-807c-8d066410e62d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:10.848060 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1202/2424] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:10 2022] POST /identity/v3/projects => generated 387 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:10.852611 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716ff38-1dc4-4496-beae-8fe859f34792 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:10.864143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716ff38-1dc4-4496-beae-8fe859f34792 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:10.922774 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3b2e6-722b-4057-a666-ea58a83418dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:10.923394 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3b2e6-722b-4057-a666-ea58a83418dd admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:10.923675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3b2e6-722b-4057-a666-ea58a83418dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:10.924213 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3b2e6-722b-4057-a666-ea58a83418dd admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:10.924994 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3b2e6-722b-4057-a666-ea58a83418dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:10.926872 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3b2e6-722b-4057-a666-ea58a83418dd admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:10.948730 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1223/2425] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:10 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:10.953605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc04ea8-08ff-495a-807c-8d066410e62d admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:10.964133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc04ea8-08ff-495a-807c-8d066410e62d admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:10.969054 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92e6cc-d11c-456b-8942-5eca6fdde7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:10.969728 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92e6cc-d11c-456b-8942-5eca6fdde7f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:10.970106 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92e6cc-d11c-456b-8942-5eca6fdde7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:10.970580 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92e6cc-d11c-456b-8942-5eca6fdde7f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:10.971397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92e6cc-d11c-456b-8942-5eca6fdde7f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:10.973164 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92e6cc-d11c-456b-8942-5eca6fdde7f5 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:10.984186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1203/2426] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:10.988245 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3b2e6-722b-4057-a666-ea58a83418dd admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:10.999026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3b2e6-722b-4057-a666-ea58a83418dd admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:11.013408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f1d40-a480-4ffa-9956-cbcdb0353fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:11.015695 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f1d40-a480-4ffa-9956-cbcdb0353fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:11.015695 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f1d40-a480-4ffa-9956-cbcdb0353fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:11.015695 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f1d40-a480-4ffa-9956-cbcdb0353fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:11.016020 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f1d40-a480-4ffa-9956-cbcdb0353fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:11.017528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f1d40-a480-4ffa-9956-cbcdb0353fb1 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:11.029723 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1224/2427] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:11.034081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92e6cc-d11c-456b-8942-5eca6fdde7f5 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:11.044372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92e6cc-d11c-456b-8942-5eca6fdde7f5 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:11.049474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1b839-c08f-46e1-9809-db312e2f42f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:11.050144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1b839-c08f-46e1-9809-db312e2f42f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:11.050422 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1b839-c08f-46e1-9809-db312e2f42f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:11.050693 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1b839-c08f-46e1-9809-db312e2f42f2 admin admin] PATH_INFO: `/v3/projects/6c5d417c0a914faca01045ab8b33a57c/users/1a00dd39bc204708a32aa0c4c4914a40/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:11.067342 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc1b839-c08f-46e1-9809-db312e2f42f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c5d417c0a914faca01045ab8b33a57c, user_id=1a00dd39bc204708a32aa0c4c4914a40, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:11.070050 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc1b839-c08f-46e1-9809-db312e2f42f2 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:11.086925 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1204/2428] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:11 2022] PUT /identity/v3/projects/6c5d417c0a914faca01045ab8b33a57c/users/1a00dd39bc204708a32aa0c4c4914a40/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:11.098113 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac46230-be23-4769-a187-3411f582a31a None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:11.098402 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac46230-be23-4769-a187-3411f582a31a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:11.098677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac46230-be23-4769-a187-3411f582a31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:11.130845 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-1ac46230-be23-4769-a187-3411f582a31a None None] MFA Rules not processed for user `1a00dd39bc204708a32aa0c4c4914a40`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:11.132264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-1ac46230-be23-4769-a187-3411f582a31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:11.205695 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1225/2429] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:11.211888 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1b839-c08f-46e1-9809-db312e2f42f2 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:11.260678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:11.284650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1b839-c08f-46e1-9809-db312e2f42f2 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:11.289871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da58518-8010-4e4d-9620-c8856ec59200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:11.290502 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da58518-8010-4e4d-9620-c8856ec59200 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:11.290930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da58518-8010-4e4d-9620-c8856ec59200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:11.291161 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da58518-8010-4e4d-9620-c8856ec59200 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:11.291948 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da58518-8010-4e4d-9620-c8856ec59200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:11.293922 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da58518-8010-4e4d-9620-c8856ec59200 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:11.326021 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1205/2430] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:11 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:19:11.330956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac46230-be23-4769-a187-3411f582a31a None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:11.343764 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac46230-be23-4769-a187-3411f582a31a None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:11.393299 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:11.417724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a67e2-27f2-4156-8ebb-07cb581f668e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:11.418405 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a67e2-27f2-4156-8ebb-07cb581f668e admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:11.418688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a67e2-27f2-4156-8ebb-07cb581f668e admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:11.418977 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a67e2-27f2-4156-8ebb-07cb581f668e admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:11.419692 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a67e2-27f2-4156-8ebb-07cb581f668e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:11.421810 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a67e2-27f2-4156-8ebb-07cb581f668e admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:11.446846 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1226/2431] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:11 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:19:11.452590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da58518-8010-4e4d-9620-c8856ec59200 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:11.466832 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da58518-8010-4e4d-9620-c8856ec59200 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:11.472827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7145c60-f528-41a3-ae48-80b83934152a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:11.473479 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7145c60-f528-41a3-ae48-80b83934152a admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:11.473754 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7145c60-f528-41a3-ae48-80b83934152a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:11.474027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7145c60-f528-41a3-ae48-80b83934152a admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:11.474733 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7145c60-f528-41a3-ae48-80b83934152a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:11.476354 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7145c60-f528-41a3-ae48-80b83934152a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:11.488737 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1206/2432] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:11.493444 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a67e2-27f2-4156-8ebb-07cb581f668e admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:11.505057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a67e2-27f2-4156-8ebb-07cb581f668e admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:11.510724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877650e-689d-49be-82dc-d89ce47a29bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:11.511207 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877650e-689d-49be-82dc-d89ce47a29bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:11.511491 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877650e-689d-49be-82dc-d89ce47a29bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:11.511809 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877650e-689d-49be-82dc-d89ce47a29bf admin admin] PATH_INFO: `/v3/projects/0055340144eb4f869487fa3d4efaf402/users/5ef1c96327d8484d875026ebe8ac6af8/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:11.530826 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877650e-689d-49be-82dc-d89ce47a29bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0055340144eb4f869487fa3d4efaf402, user_id=5ef1c96327d8484d875026ebe8ac6af8, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:11.533591 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877650e-689d-49be-82dc-d89ce47a29bf admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:11.551537 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1227/2433] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:11 2022] PUT /identity/v3/projects/0055340144eb4f869487fa3d4efaf402/users/5ef1c96327d8484d875026ebe8ac6af8/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:11.557223 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7145c60-f528-41a3-ae48-80b83934152a admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:11.569754 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7145c60-f528-41a3-ae48-80b83934152a admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:11.645005 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e24c59-9858-4c72-b388-d045a109a899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:11.645880 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e24c59-9858-4c72-b388-d045a109a899 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:11.646185 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e24c59-9858-4c72-b388-d045a109a899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:11.646935 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e24c59-9858-4c72-b388-d045a109a899 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:11.647201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e24c59-9858-4c72-b388-d045a109a899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:11.648831 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e24c59-9858-4c72-b388-d045a109a899 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:11.661072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1207/2434] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:11.665353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877650e-689d-49be-82dc-d89ce47a29bf admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:11.676947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877650e-689d-49be-82dc-d89ce47a29bf admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:11.682323 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50eb9c-f7e9-4ca7-9fae-62887f8a93a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:11.683076 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50eb9c-f7e9-4ca7-9fae-62887f8a93a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:11.683350 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50eb9c-f7e9-4ca7-9fae-62887f8a93a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:11.683619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50eb9c-f7e9-4ca7-9fae-62887f8a93a6 admin admin] PATH_INFO: `/v3/projects/0055340144eb4f869487fa3d4efaf402/users/5ef1c96327d8484d875026ebe8ac6af8/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:11.685775 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50eb9c-f7e9-4ca7-9fae-62887f8a93a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0055340144eb4f869487fa3d4efaf402, user_id=5ef1c96327d8484d875026ebe8ac6af8, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:11.688456 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50eb9c-f7e9-4ca7-9fae-62887f8a93a6 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:11.701018 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1228/2435] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:11 2022] PUT /identity/v3/projects/0055340144eb4f869487fa3d4efaf402/users/5ef1c96327d8484d875026ebe8ac6af8/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:11.711943 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b63428-e57b-4352-a3f1-aba1e4dc35f5 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:11.712322 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b63428-e57b-4352-a3f1-aba1e4dc35f5 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:11.712695 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b63428-e57b-4352-a3f1-aba1e4dc35f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:11.745948 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-b6b63428-e57b-4352-a3f1-aba1e4dc35f5 None None] MFA Rules not processed for user `5ef1c96327d8484d875026ebe8ac6af8`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:11.747360 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b6b63428-e57b-4352-a3f1-aba1e4dc35f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:11.843476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1208/2436] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:19:11.856553 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50eb9c-f7e9-4ca7-9fae-62887f8a93a6 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:11.859011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3d50eb9c-f7e9-4ca7-9fae-62887f8a93a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:11.931530 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50eb9c-f7e9-4ca7-9fae-62887f8a93a6 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:11.935646 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a549d-1d70-4446-bc6c-1e91153fd312 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:11.936378 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a549d-1d70-4446-bc6c-1e91153fd312 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:11.936777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a549d-1d70-4446-bc6c-1e91153fd312 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:11.937053 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a549d-1d70-4446-bc6c-1e91153fd312 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:11.947103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a549d-1d70-4446-bc6c-1e91153fd312 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:11.948567 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a549d-1d70-4446-bc6c-1e91153fd312 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:11.959957 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1229/2437] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:11 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:19:12.037686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b63428-e57b-4352-a3f1-aba1e4dc35f5 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:12.081243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:12.110725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b63428-e57b-4352-a3f1-aba1e4dc35f5 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:12.115640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359d939-0fd8-412a-91e0-c200da2e1e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:12.116282 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359d939-0fd8-412a-91e0-c200da2e1e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:12.116595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359d939-0fd8-412a-91e0-c200da2e1e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:12.116867 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359d939-0fd8-412a-91e0-c200da2e1e85 admin admin] PATH_INFO: `/v3/users/1a00dd39bc204708a32aa0c4c4914a40` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:12.118076 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359d939-0fd8-412a-91e0-c200da2e1e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a00dd39bc204708a32aa0c4c4914a40)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:12.120012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359d939-0fd8-412a-91e0-c200da2e1e85 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:12.185037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6359d939-0fd8-412a-91e0-c200da2e1e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a00dd39bc204708a32aa0c4c4914a40'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:12.185887 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6359d939-0fd8-412a-91e0-c200da2e1e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a00dd39bc204708a32aa0c4c4914a40'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:12.198213 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6359d939-0fd8-412a-91e0-c200da2e1e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a00dd39bc204708a32aa0c4c4914a40'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:12.199488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6359d939-0fd8-412a-91e0-c200da2e1e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a00dd39bc204708a32aa0c4c4914a40'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:12.219257 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6359d939-0fd8-412a-91e0-c200da2e1e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a00dd39bc204708a32aa0c4c4914a40'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:12.235982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1209/2438] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:12 2022] DELETE /identity/v3/users/1a00dd39bc204708a32aa0c4c4914a40 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:12.241948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0add627-21d8-4134-a559-6946de9753c5 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:12.242419 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0add627-21d8-4134-a559-6946de9753c5 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:12.242905 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0add627-21d8-4134-a559-6946de9753c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:12.268881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-f0add627-21d8-4134-a559-6946de9753c5 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:12.270253 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f0add627-21d8-4134-a559-6946de9753c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:12.342894 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1230/2439] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:12.353856 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359d939-0fd8-412a-91e0-c200da2e1e85 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:12.355901 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-6359d939-0fd8-412a-91e0-c200da2e1e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:12.459573 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359d939-0fd8-412a-91e0-c200da2e1e85 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:12.465082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f3b75-e001-4b7c-8bc1-bdea71331b69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:12.465775 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f3b75-e001-4b7c-8bc1-bdea71331b69 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:12.466353 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f3b75-e001-4b7c-8bc1-bdea71331b69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:12.466681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f3b75-e001-4b7c-8bc1-bdea71331b69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:12.482289 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f3b75-e001-4b7c-8bc1-bdea71331b69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:12.484333 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f3b75-e001-4b7c-8bc1-bdea71331b69 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:12.495580 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1210/2440] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:19:12.792187 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0add627-21d8-4134-a559-6946de9753c5 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:12.794283 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f0add627-21d8-4134-a559-6946de9753c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:12.807460 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0add627-21d8-4134-a559-6946de9753c5 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:12.813688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe473e91-8252-49e2-9b97-5b803cb8e06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:12.814460 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe473e91-8252-49e2-9b97-5b803cb8e06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:12.814813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe473e91-8252-49e2-9b97-5b803cb8e06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:12.815159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe473e91-8252-49e2-9b97-5b803cb8e06d admin admin] PATH_INFO: `/v3/projects/6c5d417c0a914faca01045ab8b33a57c` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:12.816567 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe473e91-8252-49e2-9b97-5b803cb8e06d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c5d417c0a914faca01045ab8b33a57c)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:12.818608 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe473e91-8252-49e2-9b97-5b803cb8e06d admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:12.859745 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:12.878535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fe473e91-8252-49e2-9b97-5b803cb8e06d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c5d417c0a914faca01045ab8b33a57c', 'request_id': 'req-fe473e91-8252-49e2-9b97-5b803cb8e06d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:12.887707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fe473e91-8252-49e2-9b97-5b803cb8e06d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c5d417c0a914faca01045ab8b33a57c', 'request_id': 'req-fe473e91-8252-49e2-9b97-5b803cb8e06d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:12.896562 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fe473e91-8252-49e2-9b97-5b803cb8e06d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c5d417c0a914faca01045ab8b33a57c', 'request_id': 'req-fe473e91-8252-49e2-9b97-5b803cb8e06d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:12.915203 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fe473e91-8252-49e2-9b97-5b803cb8e06d admin admin] The token cache is being invalidate because project 6c5d417c0a914faca01045ab8b33a57c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:12.915504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fe473e91-8252-49e2-9b97-5b803cb8e06d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:12.916179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fe473e91-8252-49e2-9b97-5b803cb8e06d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:12.917742 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1231/2441] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:12 2022] DELETE /identity/v3/projects/6c5d417c0a914faca01045ab8b33a57c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:12.922323 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f3b75-e001-4b7c-8bc1-bdea71331b69 service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:12.933986 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f3b75-e001-4b7c-8bc1-bdea71331b69 service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:13.012420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:13.021091 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0543fc-099d-4007-b452-6fc99d418dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:13.021834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0543fc-099d-4007-b452-6fc99d418dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:13.022588 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0543fc-099d-4007-b452-6fc99d418dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:13.023321 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0543fc-099d-4007-b452-6fc99d418dc2 admin admin] PATH_INFO: `/v3/users/5ef1c96327d8484d875026ebe8ac6af8` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:13.025725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0543fc-099d-4007-b452-6fc99d418dc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ef1c96327d8484d875026ebe8ac6af8)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:13.027991 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0543fc-099d-4007-b452-6fc99d418dc2 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:13.091704 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-da0543fc-099d-4007-b452-6fc99d418dc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ef1c96327d8484d875026ebe8ac6af8'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:13.092477 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-da0543fc-099d-4007-b452-6fc99d418dc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ef1c96327d8484d875026ebe8ac6af8'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:13.101842 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-da0543fc-099d-4007-b452-6fc99d418dc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ef1c96327d8484d875026ebe8ac6af8'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:13.102781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-da0543fc-099d-4007-b452-6fc99d418dc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ef1c96327d8484d875026ebe8ac6af8'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:13.116123 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-da0543fc-099d-4007-b452-6fc99d418dc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ef1c96327d8484d875026ebe8ac6af8'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:13.129589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1211/2442] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:12 2022] DELETE /identity/v3/users/5ef1c96327d8484d875026ebe8ac6af8 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:13.435047 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe473e91-8252-49e2-9b97-5b803cb8e06d admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:13.436829 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-fe473e91-8252-49e2-9b97-5b803cb8e06d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:13.455518 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe473e91-8252-49e2-9b97-5b803cb8e06d admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:13.532729 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ae0ae-74bc-42c3-af8e-26c0bdd42403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:13.533396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ae0ae-74bc-42c3-af8e-26c0bdd42403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:13.533677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ae0ae-74bc-42c3-af8e-26c0bdd42403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:13.533952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ae0ae-74bc-42c3-af8e-26c0bdd42403 admin admin] PATH_INFO: `/v3/projects/0055340144eb4f869487fa3d4efaf402` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:13.535156 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ae0ae-74bc-42c3-af8e-26c0bdd42403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0055340144eb4f869487fa3d4efaf402)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:13.537183 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ae0ae-74bc-42c3-af8e-26c0bdd42403 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:13.600193 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fc2ae0ae-74bc-42c3-af8e-26c0bdd42403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0055340144eb4f869487fa3d4efaf402', 'request_id': 'req-fc2ae0ae-74bc-42c3-af8e-26c0bdd42403'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:13.610092 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fc2ae0ae-74bc-42c3-af8e-26c0bdd42403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0055340144eb4f869487fa3d4efaf402', 'request_id': 'req-fc2ae0ae-74bc-42c3-af8e-26c0bdd42403'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:13.620707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fc2ae0ae-74bc-42c3-af8e-26c0bdd42403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0055340144eb4f869487fa3d4efaf402', 'request_id': 'req-fc2ae0ae-74bc-42c3-af8e-26c0bdd42403'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:13.637583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fc2ae0ae-74bc-42c3-af8e-26c0bdd42403 admin admin] The token cache is being invalidate because project 0055340144eb4f869487fa3d4efaf402 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:13.637977 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fc2ae0ae-74bc-42c3-af8e-26c0bdd42403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:13.639551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fc2ae0ae-74bc-42c3-af8e-26c0bdd42403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:13.642257 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1232/2443] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:13 2022] DELETE /identity/v3/projects/0055340144eb4f869487fa3d4efaf402 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:13.647337 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05f531-64a0-4a89-a53d-1e956a40fac3 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:13.647851 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05f531-64a0-4a89-a53d-1e956a40fac3 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:13.647955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05f531-64a0-4a89-a53d-1e956a40fac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:13.676314 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-ff05f531-64a0-4a89-a53d-1e956a40fac3 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:13.677578 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ff05f531-64a0-4a89-a53d-1e956a40fac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:13.739915 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1212/2444] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:13.747731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30521c-fd33-468b-99bc-7a2b0083197f None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:13.748017 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30521c-fd33-468b-99bc-7a2b0083197f None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:13.748295 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30521c-fd33-468b-99bc-7a2b0083197f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:13.768424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-5e30521c-fd33-468b-99bc-7a2b0083197f None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:13.769600 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5e30521c-fd33-468b-99bc-7a2b0083197f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:13.777516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1233/2445] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:13.782792 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05f531-64a0-4a89-a53d-1e956a40fac3 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:13.795007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05f531-64a0-4a89-a53d-1e956a40fac3 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:13.800067 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e7a46-7179-4bd4-adab-7f259f582087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:13.800711 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e7a46-7179-4bd4-adab-7f259f582087 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:13.800991 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e7a46-7179-4bd4-adab-7f259f582087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:13.801261 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e7a46-7179-4bd4-adab-7f259f582087 admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:13.801956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e7a46-7179-4bd4-adab-7f259f582087 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:13.802252 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e7a46-7179-4bd4-adab-7f259f582087 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:13.803891 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e7a46-7179-4bd4-adab-7f259f582087 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:13.816657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1213/2446] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:13.821114 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30521c-fd33-468b-99bc-7a2b0083197f None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:13.830925 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30521c-fd33-468b-99bc-7a2b0083197f None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:13.835580 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb563e-c287-4196-baa2-27a7c64a7229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:13.836196 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb563e-c287-4196-baa2-27a7c64a7229 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:13.836485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb563e-c287-4196-baa2-27a7c64a7229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:13.836770 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb563e-c287-4196-baa2-27a7c64a7229 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:13.837561 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb563e-c287-4196-baa2-27a7c64a7229 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:13.839503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb563e-c287-4196-baa2-27a7c64a7229 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:13.870867 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1234/2447] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:13 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:19:13.875800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e7a46-7179-4bd4-adab-7f259f582087 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:13.886493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e7a46-7179-4bd4-adab-7f259f582087 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:13.953791 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9726dc-87b4-4ad9-9528-d0fc9e271f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:13.954414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9726dc-87b4-4ad9-9528-d0fc9e271f5c admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:13.954685 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9726dc-87b4-4ad9-9528-d0fc9e271f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:13.954952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9726dc-87b4-4ad9-9528-d0fc9e271f5c admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:13.955612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9726dc-87b4-4ad9-9528-d0fc9e271f5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:13.957852 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9726dc-87b4-4ad9-9528-d0fc9e271f5c admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:13.970121 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:13.982807 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1214/2448] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:13 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:13.987007 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb563e-c287-4196-baa2-27a7c64a7229 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:13.998394 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb563e-c287-4196-baa2-27a7c64a7229 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:14.003846 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481412a-0742-4a9c-91eb-d73716e933e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:14.004698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481412a-0742-4a9c-91eb-d73716e933e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:14.004982 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481412a-0742-4a9c-91eb-d73716e933e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:14.005257 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481412a-0742-4a9c-91eb-d73716e933e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:14.006060 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481412a-0742-4a9c-91eb-d73716e933e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:14.007677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481412a-0742-4a9c-91eb-d73716e933e7 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:14.018409 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1235/2449] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:14.022664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9726dc-87b4-4ad9-9528-d0fc9e271f5c admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:14.033725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9726dc-87b4-4ad9-9528-d0fc9e271f5c admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:14.038691 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc76e7-f0d9-4f0d-90cb-d21a37db0ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:14.039350 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc76e7-f0d9-4f0d-90cb-d21a37db0ddb admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:14.039731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc76e7-f0d9-4f0d-90cb-d21a37db0ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:14.040008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc76e7-f0d9-4f0d-90cb-d21a37db0ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:14.040733 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc76e7-f0d9-4f0d-90cb-d21a37db0ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:14.042334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc76e7-f0d9-4f0d-90cb-d21a37db0ddb admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:14.054611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1215/2450] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:14.059482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481412a-0742-4a9c-91eb-d73716e933e7 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:14.071561 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481412a-0742-4a9c-91eb-d73716e933e7 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:14.076348 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04f543-2bf5-4ca4-96ac-885ea55e78cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:14.077050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04f543-2bf5-4ca4-96ac-885ea55e78cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:14.077432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04f543-2bf5-4ca4-96ac-885ea55e78cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:14.077609 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04f543-2bf5-4ca4-96ac-885ea55e78cb admin admin] PATH_INFO: `/v3/projects/78f0b5666b684ca58ec6eaa30d9be9c3/users/226ae96d223443adb1774033dcf30c7b/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:14.080897 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:14.097316 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04f543-2bf5-4ca4-96ac-885ea55e78cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=78f0b5666b684ca58ec6eaa30d9be9c3, user_id=226ae96d223443adb1774033dcf30c7b, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:14.101067 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04f543-2bf5-4ca4-96ac-885ea55e78cb admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:14.116451 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1236/2451] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:14 2022] PUT /identity/v3/projects/78f0b5666b684ca58ec6eaa30d9be9c3/users/226ae96d223443adb1774033dcf30c7b/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:14.127955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f171f5-6ef7-4377-97a1-b87f31e00f4b None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:14.128244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f171f5-6ef7-4377-97a1-b87f31e00f4b None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:14.128557 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f171f5-6ef7-4377-97a1-b87f31e00f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:14.158540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-55f171f5-6ef7-4377-97a1-b87f31e00f4b None None] MFA Rules not processed for user `226ae96d223443adb1774033dcf30c7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:14.159791 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-55f171f5-6ef7-4377-97a1-b87f31e00f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:14.247963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1216/2452] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:19:14.252903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04f543-2bf5-4ca4-96ac-885ea55e78cb admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:14.316855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04f543-2bf5-4ca4-96ac-885ea55e78cb admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:14.321417 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a063a-24b0-4861-990e-5bf1d01ddd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:14.322027 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a063a-24b0-4861-990e-5bf1d01ddd5d admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:14.322346 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a063a-24b0-4861-990e-5bf1d01ddd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:14.322617 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a063a-24b0-4861-990e-5bf1d01ddd5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:14.323272 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a063a-24b0-4861-990e-5bf1d01ddd5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:14.325252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a063a-24b0-4861-990e-5bf1d01ddd5d admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:14.353209 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1237/2453] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:14 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:14.357782 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f171f5-6ef7-4377-97a1-b87f31e00f4b None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:14.369189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f171f5-6ef7-4377-97a1-b87f31e00f4b None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:14.429554 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436d2b2-eb24-48e1-a3b8-1b3e3f09a274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:14.430167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436d2b2-eb24-48e1-a3b8-1b3e3f09a274 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:14.430443 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436d2b2-eb24-48e1-a3b8-1b3e3f09a274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:14.430707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436d2b2-eb24-48e1-a3b8-1b3e3f09a274 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:14.431356 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436d2b2-eb24-48e1-a3b8-1b3e3f09a274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:14.433294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436d2b2-eb24-48e1-a3b8-1b3e3f09a274 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:14.458731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1217/2454] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:14 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:14.460644 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a063a-24b0-4861-990e-5bf1d01ddd5d admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:14.472053 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a063a-24b0-4861-990e-5bf1d01ddd5d admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:14.477019 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5c281-2cc9-440e-9e03-32bfbc2d56ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:14.477651 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5c281-2cc9-440e-9e03-32bfbc2d56ce admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:14.477936 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5c281-2cc9-440e-9e03-32bfbc2d56ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:14.478295 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5c281-2cc9-440e-9e03-32bfbc2d56ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:14.479227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5c281-2cc9-440e-9e03-32bfbc2d56ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:14.480850 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5c281-2cc9-440e-9e03-32bfbc2d56ce admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:14.492582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1238/2455] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:14.497322 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436d2b2-eb24-48e1-a3b8-1b3e3f09a274 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:14.516067 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436d2b2-eb24-48e1-a3b8-1b3e3f09a274 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:14.524347 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d59929-3664-4335-a45c-ab08cd50c199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:14.525469 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d59929-3664-4335-a45c-ab08cd50c199 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:14.525968 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d59929-3664-4335-a45c-ab08cd50c199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:14.526465 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d59929-3664-4335-a45c-ab08cd50c199 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:14.527771 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d59929-3664-4335-a45c-ab08cd50c199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:14.531404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d59929-3664-4335-a45c-ab08cd50c199 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:14.550981 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1218/2456] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:14.556324 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5c281-2cc9-440e-9e03-32bfbc2d56ce admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:14.568184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5c281-2cc9-440e-9e03-32bfbc2d56ce admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:14.573093 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d5616-826b-4814-8867-bba2b314f10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:14.573777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d5616-826b-4814-8867-bba2b314f10c admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:14.574053 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d5616-826b-4814-8867-bba2b314f10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:14.574332 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d5616-826b-4814-8867-bba2b314f10c admin admin] PATH_INFO: `/v3/projects/1cef3dd13a9d49068a0b77960e9e3e11/users/0efb706ef29b402ebfb4429ae196aeaf/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:14.595165 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d5616-826b-4814-8867-bba2b314f10c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cef3dd13a9d49068a0b77960e9e3e11, user_id=0efb706ef29b402ebfb4429ae196aeaf, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:14.598122 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d5616-826b-4814-8867-bba2b314f10c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:14.613748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1239/2457] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:14 2022] PUT /identity/v3/projects/1cef3dd13a9d49068a0b77960e9e3e11/users/0efb706ef29b402ebfb4429ae196aeaf/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:14.624958 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f743830-a9bc-4939-8cb1-4a1301b96fdf None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:14.625244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f743830-a9bc-4939-8cb1-4a1301b96fdf None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:14.625519 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f743830-a9bc-4939-8cb1-4a1301b96fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:14.658514 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-8f743830-a9bc-4939-8cb1-4a1301b96fdf None None] MFA Rules not processed for user `0efb706ef29b402ebfb4429ae196aeaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:14.660190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-8f743830-a9bc-4939-8cb1-4a1301b96fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:14.716399 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:14.754236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1219/2458] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:14.756928 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d5616-826b-4814-8867-bba2b314f10c admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:14.865558 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d5616-826b-4814-8867-bba2b314f10c admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:14.872810 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee724bed-64ef-4def-9121-c4133a8ad252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:14.874510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee724bed-64ef-4def-9121-c4133a8ad252 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:14.874870 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee724bed-64ef-4def-9121-c4133a8ad252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:14.875358 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee724bed-64ef-4def-9121-c4133a8ad252 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:14.876899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee724bed-64ef-4def-9121-c4133a8ad252 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:14.880147 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee724bed-64ef-4def-9121-c4133a8ad252 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:14.927109 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1240/2459] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:14 2022] POST /identity/v3/projects => generated 387 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:19:14.932594 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f743830-a9bc-4939-8cb1-4a1301b96fdf None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:14.945065 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f743830-a9bc-4939-8cb1-4a1301b96fdf None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:15.014771 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d229e-04f3-416e-b130-27b6fa1e62b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:15.022263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d229e-04f3-416e-b130-27b6fa1e62b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:15.022263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d229e-04f3-416e-b130-27b6fa1e62b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:15.022263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d229e-04f3-416e-b130-27b6fa1e62b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:15.022263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d229e-04f3-416e-b130-27b6fa1e62b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:15.022263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d229e-04f3-416e-b130-27b6fa1e62b9 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:15.043977 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1220/2460] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:14 2022] POST /identity/v3/users => generated 300 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:19:15.049624 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee724bed-64ef-4def-9121-c4133a8ad252 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:15.061955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee724bed-64ef-4def-9121-c4133a8ad252 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:15.067286 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdaf6c3-c533-4f25-bcf1-d6b153587132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:15.067890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaf6c3-c533-4f25-bcf1-d6b153587132 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:15.068170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaf6c3-c533-4f25-bcf1-d6b153587132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:15.068444 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaf6c3-c533-4f25-bcf1-d6b153587132 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:15.069184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdaf6c3-c533-4f25-bcf1-d6b153587132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:15.070829 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdaf6c3-c533-4f25-bcf1-d6b153587132 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:15.084154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1241/2461] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:15.087902 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d229e-04f3-416e-b130-27b6fa1e62b9 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:15.099727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d229e-04f3-416e-b130-27b6fa1e62b9 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:15.105347 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca6940-9165-4767-ad03-4d6389cb4270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:15.106510 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca6940-9165-4767-ad03-4d6389cb4270 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:15.106717 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca6940-9165-4767-ad03-4d6389cb4270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:15.107026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca6940-9165-4767-ad03-4d6389cb4270 admin admin] PATH_INFO: `/v3/projects/983a56dfe8a34557bd78205fd40d50cd/users/617a2c47f6bd4bc8a0c3f61f7eb3b427/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:15.126819 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca6940-9165-4767-ad03-4d6389cb4270 admin admin] RBAC: Authorizing `identity:create_grant(project_id=983a56dfe8a34557bd78205fd40d50cd, user_id=617a2c47f6bd4bc8a0c3f61f7eb3b427, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:15.129596 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca6940-9165-4767-ad03-4d6389cb4270 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:15.149646 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1221/2462] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:15 2022] PUT /identity/v3/projects/983a56dfe8a34557bd78205fd40d50cd/users/617a2c47f6bd4bc8a0c3f61f7eb3b427/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:15.151084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdaf6c3-c533-4f25-bcf1-d6b153587132 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:15.163391 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdaf6c3-c533-4f25-bcf1-d6b153587132 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:15.168704 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:15.241366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510a5b6-8c9e-4a1d-9ef1-753d2690fb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:15.242159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510a5b6-8c9e-4a1d-9ef1-753d2690fb59 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:15.242486 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510a5b6-8c9e-4a1d-9ef1-753d2690fb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:15.242741 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510a5b6-8c9e-4a1d-9ef1-753d2690fb59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:15.243502 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510a5b6-8c9e-4a1d-9ef1-753d2690fb59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:15.245226 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510a5b6-8c9e-4a1d-9ef1-753d2690fb59 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:15.257517 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1242/2463] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:15.263024 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca6940-9165-4767-ad03-4d6389cb4270 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:15.276234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca6940-9165-4767-ad03-4d6389cb4270 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:15.283529 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed741b50-03be-4726-a391-c5459aa9fa4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:15.284763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed741b50-03be-4726-a391-c5459aa9fa4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:15.285342 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed741b50-03be-4726-a391-c5459aa9fa4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:15.285865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed741b50-03be-4726-a391-c5459aa9fa4d admin admin] PATH_INFO: `/v3/projects/983a56dfe8a34557bd78205fd40d50cd/users/617a2c47f6bd4bc8a0c3f61f7eb3b427/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:15.288827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed741b50-03be-4726-a391-c5459aa9fa4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=983a56dfe8a34557bd78205fd40d50cd, user_id=617a2c47f6bd4bc8a0c3f61f7eb3b427, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:15.292930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed741b50-03be-4726-a391-c5459aa9fa4d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:15.308323 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1222/2464] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:15 2022] PUT /identity/v3/projects/983a56dfe8a34557bd78205fd40d50cd/users/617a2c47f6bd4bc8a0c3f61f7eb3b427/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:15.322262 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefb9a1-7fcb-4bd6-aead-d845bdd34a81 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:15.322572 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefb9a1-7fcb-4bd6-aead-d845bdd34a81 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:15.322865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefb9a1-7fcb-4bd6-aead-d845bdd34a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:15.354414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-1fefb9a1-7fcb-4bd6-aead-d845bdd34a81 None None] MFA Rules not processed for user `617a2c47f6bd4bc8a0c3f61f7eb3b427`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:15.356054 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-1fefb9a1-7fcb-4bd6-aead-d845bdd34a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:15.446452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1243/2465] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:15.458549 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed741b50-03be-4726-a391-c5459aa9fa4d admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:15.461003 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ed741b50-03be-4726-a391-c5459aa9fa4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:15.526338 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed741b50-03be-4726-a391-c5459aa9fa4d admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:15.530846 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c0442-9ba8-47b8-b06a-7672b33fe070 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:15.531524 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c0442-9ba8-47b8-b06a-7672b33fe070 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:15.531803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c0442-9ba8-47b8-b06a-7672b33fe070 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:15.532075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c0442-9ba8-47b8-b06a-7672b33fe070 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:15.541620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c0442-9ba8-47b8-b06a-7672b33fe070 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:15.543062 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c0442-9ba8-47b8-b06a-7672b33fe070 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:15.553695 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1223/2466] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:16.151699 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fefb9a1-7fcb-4bd6-aead-d845bdd34a81 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:16.153191 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-1fefb9a1-7fcb-4bd6-aead-d845bdd34a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:16.165773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fefb9a1-7fcb-4bd6-aead-d845bdd34a81 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:16.169717 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53a9d1-47e6-4531-b53e-4af2e17502e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:16.170342 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53a9d1-47e6-4531-b53e-4af2e17502e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:16.170621 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53a9d1-47e6-4531-b53e-4af2e17502e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:16.170896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53a9d1-47e6-4531-b53e-4af2e17502e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:16.233235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53a9d1-47e6-4531-b53e-4af2e17502e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:16.234662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53a9d1-47e6-4531-b53e-4af2e17502e9 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:16.243881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1244/2467] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:17.693423 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c0442-9ba8-47b8-b06a-7672b33fe070 service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:17.701372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-384c0442-9ba8-47b8-b06a-7672b33fe070 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:17.717851 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a5095-513a-4ee0-90ad-c93e1c9cf81c None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:17.718357 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a5095-513a-4ee0-90ad-c93e1c9cf81c None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:17.718458 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a5095-513a-4ee0-90ad-c93e1c9cf81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:17.746064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-df6a5095-513a-4ee0-90ad-c93e1c9cf81c None None] MFA Rules not processed for user `4370f8f474c2446e989d0d8c5b75e87a`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:17.747308 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-df6a5095-513a-4ee0-90ad-c93e1c9cf81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:17.777405 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:17.789619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:17.827439 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1245/2468] 10.222.0.93 () {58 vars in 1009 bytes} [Mon Aug 1 15:19:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:19:17.835277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a5095-513a-4ee0-90ad-c93e1c9cf81c None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:17.837861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-df6a5095-513a-4ee0-90ad-c93e1c9cf81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:17.841902 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c0442-9ba8-47b8-b06a-7672b33fe070 service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:17.851066 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa194e-e7a2-4708-8407-5b486865f9c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:17.851843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa194e-e7a2-4708-8407-5b486865f9c8 service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:17.852179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa194e-e7a2-4708-8407-5b486865f9c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:17.852408 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa194e-e7a2-4708-8407-5b486865f9c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:17.855837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-55fa194e-e7a2-4708-8407-5b486865f9c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:17.892586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa194e-e7a2-4708-8407-5b486865f9c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:17.894473 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa194e-e7a2-4708-8407-5b486865f9c8 service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:17.920561 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1224/2469] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:18.000833 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a5095-513a-4ee0-90ad-c93e1c9cf81c None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:18.008161 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ee2d6-f7fb-4f92-a5c8-119f744248ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4df37b02152c48509243b73ebbf49d8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:18.009101 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ee2d6-f7fb-4f92-a5c8-119f744248ad service swift] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:18.009396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ee2d6-f7fb-4f92-a5c8-119f744248ad service swift] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:18.009773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ee2d6-f7fb-4f92-a5c8-119f744248ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:18.019638 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ee2d6-f7fb-4f92-a5c8-119f744248ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:18.021231 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ee2d6-f7fb-4f92-a5c8-119f744248ad service swift] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:18.033157 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1246/2470] 10.222.0.93 () {60 vars in 1384 bytes} [Mon Aug 1 15:19:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 15:19:20.798042 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa194e-e7a2-4708-8407-5b486865f9c8 service nova] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:20.810819 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa194e-e7a2-4708-8407-5b486865f9c8 service nova] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:20.815834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf49e58-d8c6-470a-ae5f-2fe7becddae4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:20.817145 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf49e58-d8c6-470a-ae5f-2fe7becddae4 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:20.817460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf49e58-d8c6-470a-ae5f-2fe7becddae4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:20.817827 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf49e58-d8c6-470a-ae5f-2fe7becddae4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:20.908616 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf49e58-d8c6-470a-ae5f-2fe7becddae4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:20.909660 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf49e58-d8c6-470a-ae5f-2fe7becddae4 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:20.930159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1225/2471] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:19:22.724769 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ee2d6-f7fb-4f92-a5c8-119f744248ad service swift] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:22.726373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3f5ee2d6-f7fb-4f92-a5c8-119f744248ad service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:22.803345 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ee2d6-f7fb-4f92-a5c8-119f744248ad service swift] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:22.807472 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f21acd-2f50-4116-bd0e-d0c0bc10ff32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:22.808229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f21acd-2f50-4116-bd0e-d0c0bc10ff32 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:22.808384 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f21acd-2f50-4116-bd0e-d0c0bc10ff32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:22.809441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f21acd-2f50-4116-bd0e-d0c0bc10ff32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:22.811962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-71f21acd-2f50-4116-bd0e-d0c0bc10ff32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:22.898255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f21acd-2f50-4116-bd0e-d0c0bc10ff32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:22.899711 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f21acd-2f50-4116-bd0e-d0c0bc10ff32 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:22.917884 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1247/2472] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:22 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:24.032080 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf49e58-d8c6-470a-ae5f-2fe7becddae4 service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:24.124921 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf49e58-d8c6-470a-ae5f-2fe7becddae4 service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:24.129769 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e39ee-1290-4150-88ec-db28815cb830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:24.130417 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e39ee-1290-4150-88ec-db28815cb830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:24.130699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e39ee-1290-4150-88ec-db28815cb830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:24.131016 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e39ee-1290-4150-88ec-db28815cb830 admin admin] PATH_INFO: `/v3/users/226ae96d223443adb1774033dcf30c7b` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:24.133121 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e39ee-1290-4150-88ec-db28815cb830 admin admin] RBAC: Authorizing `identity:delete_user(user_id=226ae96d223443adb1774033dcf30c7b)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:24.135391 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e39ee-1290-4150-88ec-db28815cb830 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:24.204105 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-053e39ee-1290-4150-88ec-db28815cb830 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '226ae96d223443adb1774033dcf30c7b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:24.205320 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-053e39ee-1290-4150-88ec-db28815cb830 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '226ae96d223443adb1774033dcf30c7b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:24.218669 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-053e39ee-1290-4150-88ec-db28815cb830 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '226ae96d223443adb1774033dcf30c7b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:24.219447 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-053e39ee-1290-4150-88ec-db28815cb830 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '226ae96d223443adb1774033dcf30c7b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:24.226001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:24.246696 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-053e39ee-1290-4150-88ec-db28815cb830 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '226ae96d223443adb1774033dcf30c7b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:24.246696 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1226/2473] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:24 2022] DELETE /identity/v3/users/226ae96d223443adb1774033dcf30c7b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:24.268808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3d117-b437-4b5f-b44c-a9b3aafe6a8d None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:24.268808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3d117-b437-4b5f-b44c-a9b3aafe6a8d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:24.268808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3d117-b437-4b5f-b44c-a9b3aafe6a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:24.288449 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-e0d3d117-b437-4b5f-b44c-a9b3aafe6a8d None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:24.289757 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e0d3d117-b437-4b5f-b44c-a9b3aafe6a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:24.300085 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:24.396061 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1248/2474] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:24.410630 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e39ee-1290-4150-88ec-db28815cb830 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:24.412426 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-053e39ee-1290-4150-88ec-db28815cb830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:24.598036 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e39ee-1290-4150-88ec-db28815cb830 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:24.602188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c7f73-4e89-4766-a0ad-0c894d5426ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:24.602865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c7f73-4e89-4766-a0ad-0c894d5426ba service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:24.603709 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c7f73-4e89-4766-a0ad-0c894d5426ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:24.603709 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c7f73-4e89-4766-a0ad-0c894d5426ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:24.636871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c7f73-4e89-4766-a0ad-0c894d5426ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:24.639579 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c7f73-4e89-4766-a0ad-0c894d5426ba service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:24.652342 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1227/2475] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:25.032153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3d117-b437-4b5f-b44c-a9b3aafe6a8d None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:25.033854 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e0d3d117-b437-4b5f-b44c-a9b3aafe6a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:25.057666 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3d117-b437-4b5f-b44c-a9b3aafe6a8d None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:25.065919 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d7a07-6563-4f76-94b9-358b862d92bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:25.067767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d7a07-6563-4f76-94b9-358b862d92bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:25.068684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d7a07-6563-4f76-94b9-358b862d92bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:25.070108 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d7a07-6563-4f76-94b9-358b862d92bd admin admin] PATH_INFO: `/v3/projects/78f0b5666b684ca58ec6eaa30d9be9c3` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:25.072798 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d7a07-6563-4f76-94b9-358b862d92bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=78f0b5666b684ca58ec6eaa30d9be9c3)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:25.075385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d7a07-6563-4f76-94b9-358b862d92bd admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:25.146783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6a4d7a07-6563-4f76-94b9-358b862d92bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78f0b5666b684ca58ec6eaa30d9be9c3', 'request_id': 'req-6a4d7a07-6563-4f76-94b9-358b862d92bd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:25.158287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6a4d7a07-6563-4f76-94b9-358b862d92bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78f0b5666b684ca58ec6eaa30d9be9c3', 'request_id': 'req-6a4d7a07-6563-4f76-94b9-358b862d92bd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:25.171032 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6a4d7a07-6563-4f76-94b9-358b862d92bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78f0b5666b684ca58ec6eaa30d9be9c3', 'request_id': 'req-6a4d7a07-6563-4f76-94b9-358b862d92bd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:25.184990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6a4d7a07-6563-4f76-94b9-358b862d92bd admin admin] The token cache is being invalidate because project 78f0b5666b684ca58ec6eaa30d9be9c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:25.185310 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6a4d7a07-6563-4f76-94b9-358b862d92bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:25.186896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6a4d7a07-6563-4f76-94b9-358b862d92bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:25.189108 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1249/2476] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:25 2022] DELETE /identity/v3/projects/78f0b5666b684ca58ec6eaa30d9be9c3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:25.193185 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c7f73-4e89-4766-a0ad-0c894d5426ba service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:25.220071 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c7f73-4e89-4766-a0ad-0c894d5426ba service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:25.369256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9d6f5-dce5-4efe-b068-c169109ce6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:25.370251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9d6f5-dce5-4efe-b068-c169109ce6ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:25.370718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9d6f5-dce5-4efe-b068-c169109ce6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:25.371411 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9d6f5-dce5-4efe-b068-c169109ce6ba admin admin] PATH_INFO: `/v3/users/0efb706ef29b402ebfb4429ae196aeaf` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:25.373936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9d6f5-dce5-4efe-b068-c169109ce6ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=0efb706ef29b402ebfb4429ae196aeaf)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:25.377918 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9d6f5-dce5-4efe-b068-c169109ce6ba admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:25.465590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-ecc9d6f5-dce5-4efe-b068-c169109ce6ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0efb706ef29b402ebfb4429ae196aeaf'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:25.466308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-ecc9d6f5-dce5-4efe-b068-c169109ce6ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0efb706ef29b402ebfb4429ae196aeaf'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:25.481798 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-ecc9d6f5-dce5-4efe-b068-c169109ce6ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0efb706ef29b402ebfb4429ae196aeaf'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:25.483146 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-ecc9d6f5-dce5-4efe-b068-c169109ce6ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0efb706ef29b402ebfb4429ae196aeaf'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:25.500828 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-ecc9d6f5-dce5-4efe-b068-c169109ce6ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0efb706ef29b402ebfb4429ae196aeaf'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:25.517197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:25.524795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1228/2477] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:25 2022] DELETE /identity/v3/users/0efb706ef29b402ebfb4429ae196aeaf => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:25.955692 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d7a07-6563-4f76-94b9-358b862d92bd admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:25.958403 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-6a4d7a07-6563-4f76-94b9-358b862d92bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:25.981900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d7a07-6563-4f76-94b9-358b862d92bd admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:26.113876 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88645772-13e6-4e10-a75d-4fae0e695251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:26.114536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88645772-13e6-4e10-a75d-4fae0e695251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:26.114814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88645772-13e6-4e10-a75d-4fae0e695251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:26.115253 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88645772-13e6-4e10-a75d-4fae0e695251 admin admin] PATH_INFO: `/v3/projects/1cef3dd13a9d49068a0b77960e9e3e11` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:26.116850 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88645772-13e6-4e10-a75d-4fae0e695251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cef3dd13a9d49068a0b77960e9e3e11)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:26.118967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88645772-13e6-4e10-a75d-4fae0e695251 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:26.210127 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:26.240642 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-88645772-13e6-4e10-a75d-4fae0e695251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cef3dd13a9d49068a0b77960e9e3e11', 'request_id': 'req-88645772-13e6-4e10-a75d-4fae0e695251'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:26.256872 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-88645772-13e6-4e10-a75d-4fae0e695251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cef3dd13a9d49068a0b77960e9e3e11', 'request_id': 'req-88645772-13e6-4e10-a75d-4fae0e695251'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:26.272195 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-88645772-13e6-4e10-a75d-4fae0e695251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cef3dd13a9d49068a0b77960e9e3e11', 'request_id': 'req-88645772-13e6-4e10-a75d-4fae0e695251'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:26.294520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-88645772-13e6-4e10-a75d-4fae0e695251 admin admin] The token cache is being invalidate because project 1cef3dd13a9d49068a0b77960e9e3e11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:26.295766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-88645772-13e6-4e10-a75d-4fae0e695251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:26.296812 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-88645772-13e6-4e10-a75d-4fae0e695251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:26.302113 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1250/2478] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:25 2022] DELETE /identity/v3/projects/1cef3dd13a9d49068a0b77960e9e3e11 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:26.306055 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9d6f5-dce5-4efe-b068-c169109ce6ba admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:26.307621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ecc9d6f5-dce5-4efe-b068-c169109ce6ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:26.338482 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9d6f5-dce5-4efe-b068-c169109ce6ba admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:26.484435 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7a075-9526-4a8b-a27c-37c2af1e4775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:26.484999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7a075-9526-4a8b-a27c-37c2af1e4775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:26.485314 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7a075-9526-4a8b-a27c-37c2af1e4775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:26.485632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7a075-9526-4a8b-a27c-37c2af1e4775 admin admin] PATH_INFO: `/v3/users/617a2c47f6bd4bc8a0c3f61f7eb3b427` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:26.493947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b7a075-9526-4a8b-a27c-37c2af1e4775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=617a2c47f6bd4bc8a0c3f61f7eb3b427)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:26.495944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b7a075-9526-4a8b-a27c-37c2af1e4775 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:26.613226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-23b7a075-9526-4a8b-a27c-37c2af1e4775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '617a2c47f6bd4bc8a0c3f61f7eb3b427'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:26.614448 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-23b7a075-9526-4a8b-a27c-37c2af1e4775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '617a2c47f6bd4bc8a0c3f61f7eb3b427'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:26.629423 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-23b7a075-9526-4a8b-a27c-37c2af1e4775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '617a2c47f6bd4bc8a0c3f61f7eb3b427'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:26.631638 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-23b7a075-9526-4a8b-a27c-37c2af1e4775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '617a2c47f6bd4bc8a0c3f61f7eb3b427'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:26.663504 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-23b7a075-9526-4a8b-a27c-37c2af1e4775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '617a2c47f6bd4bc8a0c3f61f7eb3b427'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:26.687135 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1229/2479] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:26 2022] DELETE /identity/v3/users/617a2c47f6bd4bc8a0c3f61f7eb3b427 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:27.254856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88645772-13e6-4e10-a75d-4fae0e695251 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:27.259661 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-88645772-13e6-4e10-a75d-4fae0e695251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:27.285735 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88645772-13e6-4e10-a75d-4fae0e695251 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:27.452937 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a42ed3-e47a-4b46-9d21-ad96fcaa49cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:27.453512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a42ed3-e47a-4b46-9d21-ad96fcaa49cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:27.453792 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a42ed3-e47a-4b46-9d21-ad96fcaa49cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:27.454067 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a42ed3-e47a-4b46-9d21-ad96fcaa49cd admin admin] PATH_INFO: `/v3/projects/983a56dfe8a34557bd78205fd40d50cd` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:27.455270 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a42ed3-e47a-4b46-9d21-ad96fcaa49cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=983a56dfe8a34557bd78205fd40d50cd)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:27.457266 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a42ed3-e47a-4b46-9d21-ad96fcaa49cd admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:27.524594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-60a42ed3-e47a-4b46-9d21-ad96fcaa49cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '983a56dfe8a34557bd78205fd40d50cd', 'request_id': 'req-60a42ed3-e47a-4b46-9d21-ad96fcaa49cd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:27.536519 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-60a42ed3-e47a-4b46-9d21-ad96fcaa49cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '983a56dfe8a34557bd78205fd40d50cd', 'request_id': 'req-60a42ed3-e47a-4b46-9d21-ad96fcaa49cd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:27.560159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-60a42ed3-e47a-4b46-9d21-ad96fcaa49cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '983a56dfe8a34557bd78205fd40d50cd', 'request_id': 'req-60a42ed3-e47a-4b46-9d21-ad96fcaa49cd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:27.582687 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-60a42ed3-e47a-4b46-9d21-ad96fcaa49cd admin admin] The token cache is being invalidate because project 983a56dfe8a34557bd78205fd40d50cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:27.582965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-60a42ed3-e47a-4b46-9d21-ad96fcaa49cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:27.583834 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-60a42ed3-e47a-4b46-9d21-ad96fcaa49cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:27.587292 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1251/2480] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:27 2022] DELETE /identity/v3/projects/983a56dfe8a34557bd78205fd40d50cd => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:27.595428 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838eb904-9049-4108-be25-edebd87fa0f3 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:27.595825 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838eb904-9049-4108-be25-edebd87fa0f3 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:27.595919 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838eb904-9049-4108-be25-edebd87fa0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:27.640718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-838eb904-9049-4108-be25-edebd87fa0f3 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:27.640718 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-838eb904-9049-4108-be25-edebd87fa0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:27.743266 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1230/2481] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:27.745301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130319a-3807-46d6-9723-74ff9defdbaf None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:27.745674 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130319a-3807-46d6-9723-74ff9defdbaf None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:27.746104 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130319a-3807-46d6-9723-74ff9defdbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:27.801308 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-9130319a-3807-46d6-9723-74ff9defdbaf None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:27.802619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-9130319a-3807-46d6-9723-74ff9defdbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:27.820992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1252/2482] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:27.822476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838eb904-9049-4108-be25-edebd87fa0f3 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:27.833768 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838eb904-9049-4108-be25-edebd87fa0f3 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:27.842699 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54f8a5-cf9c-4999-869d-c46abd9c0764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:27.843636 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54f8a5-cf9c-4999-869d-c46abd9c0764 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:27.844342 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54f8a5-cf9c-4999-869d-c46abd9c0764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:27.844828 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54f8a5-cf9c-4999-869d-c46abd9c0764 admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:27.845885 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54f8a5-cf9c-4999-869d-c46abd9c0764 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:27.846519 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54f8a5-cf9c-4999-869d-c46abd9c0764 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:27.849081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54f8a5-cf9c-4999-869d-c46abd9c0764 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:27.878879 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1231/2483] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:27.879534 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130319a-3807-46d6-9723-74ff9defdbaf None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:27.894797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130319a-3807-46d6-9723-74ff9defdbaf None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:27.905716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d2b96-0fb9-48eb-98fb-72f252c2873a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:27.906656 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d2b96-0fb9-48eb-98fb-72f252c2873a admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:27.907042 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d2b96-0fb9-48eb-98fb-72f252c2873a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:27.907426 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d2b96-0fb9-48eb-98fb-72f252c2873a admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:27.908215 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d2b96-0fb9-48eb-98fb-72f252c2873a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:27.910482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d2b96-0fb9-48eb-98fb-72f252c2873a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:27.964556 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1253/2484] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:27 2022] POST /identity/v3/projects => generated 379 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:19:27.966026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54f8a5-cf9c-4999-869d-c46abd9c0764 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:27.987506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54f8a5-cf9c-4999-869d-c46abd9c0764 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:27.994800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:28.091034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5104f3-8c54-4bcb-961d-bb5b2e3413e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:28.092321 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5104f3-8c54-4bcb-961d-bb5b2e3413e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:28.092608 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5104f3-8c54-4bcb-961d-bb5b2e3413e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:28.093034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5104f3-8c54-4bcb-961d-bb5b2e3413e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:28.093752 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5104f3-8c54-4bcb-961d-bb5b2e3413e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:28.096405 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5104f3-8c54-4bcb-961d-bb5b2e3413e7 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:28.137294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1232/2485] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:27 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:28.142336 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d2b96-0fb9-48eb-98fb-72f252c2873a admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:28.165382 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d2b96-0fb9-48eb-98fb-72f252c2873a admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:28.178497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068d8de-9133-47a8-87a3-d322010857a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:28.179756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068d8de-9133-47a8-87a3-d322010857a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:28.180450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068d8de-9133-47a8-87a3-d322010857a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:28.181005 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068d8de-9133-47a8-87a3-d322010857a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:28.183283 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068d8de-9133-47a8-87a3-d322010857a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:28.188703 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068d8de-9133-47a8-87a3-d322010857a3 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:28.220357 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1254/2486] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:28.224796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5104f3-8c54-4bcb-961d-bb5b2e3413e7 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:28.241117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5104f3-8c54-4bcb-961d-bb5b2e3413e7 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:28.246679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2a56a-d79b-404a-8f28-27d49eeb3792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:28.247389 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2a56a-d79b-404a-8f28-27d49eeb3792 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:28.247772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2a56a-d79b-404a-8f28-27d49eeb3792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:28.248158 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2a56a-d79b-404a-8f28-27d49eeb3792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:28.248975 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2a56a-d79b-404a-8f28-27d49eeb3792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:28.250710 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2a56a-d79b-404a-8f28-27d49eeb3792 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:28.267127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1233/2487] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:28.271394 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068d8de-9133-47a8-87a3-d322010857a3 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:28.286811 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068d8de-9133-47a8-87a3-d322010857a3 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:28.292245 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d6b50-91ea-48ed-ae29-32c5e914511e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:28.292983 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d6b50-91ea-48ed-ae29-32c5e914511e admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:28.293397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d6b50-91ea-48ed-ae29-32c5e914511e admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:28.293673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d6b50-91ea-48ed-ae29-32c5e914511e admin admin] PATH_INFO: `/v3/projects/ab3be2f13cb24652b1f6e484d31f335d/users/00fd42c83e1240f2bebfd43f755970b3/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:28.316502 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d6b50-91ea-48ed-ae29-32c5e914511e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab3be2f13cb24652b1f6e484d31f335d, user_id=00fd42c83e1240f2bebfd43f755970b3, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:28.319264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d6b50-91ea-48ed-ae29-32c5e914511e admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:28.322193 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:28.344561 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1255/2488] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:28 2022] PUT /identity/v3/projects/ab3be2f13cb24652b1f6e484d31f335d/users/00fd42c83e1240f2bebfd43f755970b3/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:28.356742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d23d3-5c00-4a84-bec6-9417e3fbbf20 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:28.357030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d23d3-5c00-4a84-bec6-9417e3fbbf20 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:28.357339 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d23d3-5c00-4a84-bec6-9417e3fbbf20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:28.391700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-814d23d3-5c00-4a84-bec6-9417e3fbbf20 None None] MFA Rules not processed for user `00fd42c83e1240f2bebfd43f755970b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:28.394025 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-814d23d3-5c00-4a84-bec6-9417e3fbbf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:28.503646 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1234/2489] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:28.506861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d6b50-91ea-48ed-ae29-32c5e914511e admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:28.601719 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d6b50-91ea-48ed-ae29-32c5e914511e admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:28.610601 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88466eb5-9047-4adf-bfa0-9fa7aa5870f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:28.612029 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88466eb5-9047-4adf-bfa0-9fa7aa5870f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:28.612402 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88466eb5-9047-4adf-bfa0-9fa7aa5870f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:28.612836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88466eb5-9047-4adf-bfa0-9fa7aa5870f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:28.613713 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88466eb5-9047-4adf-bfa0-9fa7aa5870f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:28.616427 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88466eb5-9047-4adf-bfa0-9fa7aa5870f9 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:28.671176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1256/2490] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:28 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:28.675597 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d23d3-5c00-4a84-bec6-9417e3fbbf20 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:28.690698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d23d3-5c00-4a84-bec6-9417e3fbbf20 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:28.762109 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235559d-b48b-4a78-b3b7-65e0bc917026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:28.762671 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235559d-b48b-4a78-b3b7-65e0bc917026 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:28.762954 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235559d-b48b-4a78-b3b7-65e0bc917026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:28.763228 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235559d-b48b-4a78-b3b7-65e0bc917026 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:28.763983 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235559d-b48b-4a78-b3b7-65e0bc917026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:28.766112 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235559d-b48b-4a78-b3b7-65e0bc917026 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:28.803441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1235/2491] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:28 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:28.808043 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88466eb5-9047-4adf-bfa0-9fa7aa5870f9 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:28.818935 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88466eb5-9047-4adf-bfa0-9fa7aa5870f9 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:28.823972 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49512d48-4110-4cc0-9903-393b3d3dd63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:28.824656 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49512d48-4110-4cc0-9903-393b3d3dd63a admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:28.824996 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49512d48-4110-4cc0-9903-393b3d3dd63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:28.825276 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49512d48-4110-4cc0-9903-393b3d3dd63a admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:28.826012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49512d48-4110-4cc0-9903-393b3d3dd63a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:28.827784 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49512d48-4110-4cc0-9903-393b3d3dd63a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:28.846359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1257/2492] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:28.849609 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235559d-b48b-4a78-b3b7-65e0bc917026 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:28.858478 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:28.864113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235559d-b48b-4a78-b3b7-65e0bc917026 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:28.874044 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ea8a8-bbde-4892-ad05-6f5121ee2d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:28.875181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ea8a8-bbde-4892-ad05-6f5121ee2d28 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:28.875679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ea8a8-bbde-4892-ad05-6f5121ee2d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:28.875829 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ea8a8-bbde-4892-ad05-6f5121ee2d28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:28.876514 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ea8a8-bbde-4892-ad05-6f5121ee2d28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:28.879438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ea8a8-bbde-4892-ad05-6f5121ee2d28 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:28.895773 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1236/2493] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:19:28.903676 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49512d48-4110-4cc0-9903-393b3d3dd63a admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:28.917474 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49512d48-4110-4cc0-9903-393b3d3dd63a admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:28.923235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce6c59-4ec3-491c-9ff9-8dad232fef25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:28.924070 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce6c59-4ec3-491c-9ff9-8dad232fef25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:28.924724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce6c59-4ec3-491c-9ff9-8dad232fef25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:28.924724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce6c59-4ec3-491c-9ff9-8dad232fef25 admin admin] PATH_INFO: `/v3/projects/42b50839e1164e43b3f05e137bc3cbed/users/773ff484c9874ca6afed9baa6d604ad6/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:28.941565 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce6c59-4ec3-491c-9ff9-8dad232fef25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b50839e1164e43b3f05e137bc3cbed, user_id=773ff484c9874ca6afed9baa6d604ad6, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:28.943780 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce6c59-4ec3-491c-9ff9-8dad232fef25 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:28.966007 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1258/2494] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:28 2022] PUT /identity/v3/projects/42b50839e1164e43b3f05e137bc3cbed/users/773ff484c9874ca6afed9baa6d604ad6/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:29.038067 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a112ff8-3fec-4e94-85fa-1233c3b665bd None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:29.038327 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a112ff8-3fec-4e94-85fa-1233c3b665bd None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:29.038608 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a112ff8-3fec-4e94-85fa-1233c3b665bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:29.076892 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-3a112ff8-3fec-4e94-85fa-1233c3b665bd None None] MFA Rules not processed for user `773ff484c9874ca6afed9baa6d604ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:29.078158 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-3a112ff8-3fec-4e94-85fa-1233c3b665bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:29.249496 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1237/2495] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:29.256188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce6c59-4ec3-491c-9ff9-8dad232fef25 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:29.365139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce6c59-4ec3-491c-9ff9-8dad232fef25 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:29.371898 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c97440-c7c2-409d-a1cd-cbd911db691e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:29.372692 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c97440-c7c2-409d-a1cd-cbd911db691e admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:29.373620 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c97440-c7c2-409d-a1cd-cbd911db691e admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:29.374078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c97440-c7c2-409d-a1cd-cbd911db691e admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:29.374745 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c97440-c7c2-409d-a1cd-cbd911db691e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:29.377211 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c97440-c7c2-409d-a1cd-cbd911db691e admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:29.433863 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1259/2496] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:29 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:29.435684 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a112ff8-3fec-4e94-85fa-1233c3b665bd None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:29.460119 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a112ff8-3fec-4e94-85fa-1233c3b665bd None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:29.571117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e59c35-d0a5-4ff6-a3d5-c978855d2bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:29.571938 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e59c35-d0a5-4ff6-a3d5-c978855d2bed admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:29.572372 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e59c35-d0a5-4ff6-a3d5-c978855d2bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:29.572757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e59c35-d0a5-4ff6-a3d5-c978855d2bed admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:29.573507 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e59c35-d0a5-4ff6-a3d5-c978855d2bed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:29.575595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e59c35-d0a5-4ff6-a3d5-c978855d2bed admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:29.607992 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1238/2497] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:29 2022] POST /identity/v3/users => generated 295 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:29.608917 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c97440-c7c2-409d-a1cd-cbd911db691e admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:29.627516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c97440-c7c2-409d-a1cd-cbd911db691e admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:29.633692 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a5790-72ea-4552-9393-3bcf8593fcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:29.634304 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a5790-72ea-4552-9393-3bcf8593fcbb admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:29.634658 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a5790-72ea-4552-9393-3bcf8593fcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:29.635051 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a5790-72ea-4552-9393-3bcf8593fcbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:29.635831 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a5790-72ea-4552-9393-3bcf8593fcbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:29.637722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a5790-72ea-4552-9393-3bcf8593fcbb admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:29.655080 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e59c35-d0a5-4ff6-a3d5-c978855d2bed admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:29.662315 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1260/2498] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:29.695612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e59c35-d0a5-4ff6-a3d5-c978855d2bed admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:29.703932 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd4522-8232-4d6d-83c3-3525a70993cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:29.704777 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd4522-8232-4d6d-83c3-3525a70993cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:29.705178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd4522-8232-4d6d-83c3-3525a70993cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:29.705521 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd4522-8232-4d6d-83c3-3525a70993cd admin admin] PATH_INFO: `/v3/projects/e9670185ddb9482cbe57627a5f310c65/users/5ea620fa07ab457fb5302600e2cb05bc/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:29.726381 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd4522-8232-4d6d-83c3-3525a70993cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9670185ddb9482cbe57627a5f310c65, user_id=5ea620fa07ab457fb5302600e2cb05bc, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:29.730974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd4522-8232-4d6d-83c3-3525a70993cd admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:29.763917 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a5790-72ea-4552-9393-3bcf8593fcbb admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:29.764142 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1239/2499] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:29 2022] PUT /identity/v3/projects/e9670185ddb9482cbe57627a5f310c65/users/5ea620fa07ab457fb5302600e2cb05bc/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:29.784977 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a5790-72ea-4552-9393-3bcf8593fcbb admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:29.802983 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:29.881500 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301226f-521d-4fd0-abef-8d75f330a9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:29.882437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301226f-521d-4fd0-abef-8d75f330a9a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:29.883073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301226f-521d-4fd0-abef-8d75f330a9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:29.884256 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301226f-521d-4fd0-abef-8d75f330a9a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:29.885173 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301226f-521d-4fd0-abef-8d75f330a9a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:29.886970 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301226f-521d-4fd0-abef-8d75f330a9a7 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:29.898927 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1261/2500] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:19:29.903729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd4522-8232-4d6d-83c3-3525a70993cd admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:29.923170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd4522-8232-4d6d-83c3-3525a70993cd admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:29.928846 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57154e-912b-4962-a7cb-456e089846ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:29.929599 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57154e-912b-4962-a7cb-456e089846ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:29.929952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57154e-912b-4962-a7cb-456e089846ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:29.930288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57154e-912b-4962-a7cb-456e089846ff admin admin] PATH_INFO: `/v3/projects/e9670185ddb9482cbe57627a5f310c65/users/5ea620fa07ab457fb5302600e2cb05bc/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:29.932427 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57154e-912b-4962-a7cb-456e089846ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9670185ddb9482cbe57627a5f310c65, user_id=5ea620fa07ab457fb5302600e2cb05bc, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:29.935080 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57154e-912b-4962-a7cb-456e089846ff admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:29.949444 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1240/2501] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:29 2022] PUT /identity/v3/projects/e9670185ddb9482cbe57627a5f310c65/users/5ea620fa07ab457fb5302600e2cb05bc/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:29.960398 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ddad4-c8f0-41cf-be81-dbee429bb73f None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:29.960841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ddad4-c8f0-41cf-be81-dbee429bb73f None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:29.960964 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ddad4-c8f0-41cf-be81-dbee429bb73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:29.999820 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-0f7ddad4-c8f0-41cf-be81-dbee429bb73f None None] MFA Rules not processed for user `5ea620fa07ab457fb5302600e2cb05bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:30.001349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0f7ddad4-c8f0-41cf-be81-dbee429bb73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:30.098701 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1262/2502] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:30.112667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57154e-912b-4962-a7cb-456e089846ff admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:30.114601 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-bf57154e-912b-4962-a7cb-456e089846ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:30.161447 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:30.182930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57154e-912b-4962-a7cb-456e089846ff admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:30.187602 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030c66e-d81b-4124-8ad4-77bc4965056b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:30.188242 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030c66e-d81b-4124-8ad4-77bc4965056b service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:30.188589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030c66e-d81b-4124-8ad4-77bc4965056b service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:30.188982 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030c66e-d81b-4124-8ad4-77bc4965056b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:30.199569 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6030c66e-d81b-4124-8ad4-77bc4965056b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:30.201137 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6030c66e-d81b-4124-8ad4-77bc4965056b service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:30.213738 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1241/2503] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:30.265866 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ddad4-c8f0-41cf-be81-dbee429bb73f None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:30.267532 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0f7ddad4-c8f0-41cf-be81-dbee429bb73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:30.281922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ddad4-c8f0-41cf-be81-dbee429bb73f None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:30.286398 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ebbcdf-4a27-4bb0-a6a0-4e01893a4bd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:30.287051 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebbcdf-4a27-4bb0-a6a0-4e01893a4bd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:30.287335 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebbcdf-4a27-4bb0-a6a0-4e01893a4bd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:30.287612 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebbcdf-4a27-4bb0-a6a0-4e01893a4bd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:30.349262 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ebbcdf-4a27-4bb0-a6a0-4e01893a4bd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:30.350692 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ebbcdf-4a27-4bb0-a6a0-4e01893a4bd3 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:30.360771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1263/2504] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:33.360931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030c66e-d81b-4124-8ad4-77bc4965056b service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:33.449057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030c66e-d81b-4124-8ad4-77bc4965056b service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:33.454128 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945f9d6-f1d3-4a9f-bb58-eec867b8b543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:33.454783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945f9d6-f1d3-4a9f-bb58-eec867b8b543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:33.455072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945f9d6-f1d3-4a9f-bb58-eec867b8b543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:33.455382 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945f9d6-f1d3-4a9f-bb58-eec867b8b543 admin admin] PATH_INFO: `/v3/users/00fd42c83e1240f2bebfd43f755970b3` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:33.456560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f945f9d6-f1d3-4a9f-bb58-eec867b8b543 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00fd42c83e1240f2bebfd43f755970b3)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:33.458502 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f945f9d6-f1d3-4a9f-bb58-eec867b8b543 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:33.528656 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-f945f9d6-f1d3-4a9f-bb58-eec867b8b543 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00fd42c83e1240f2bebfd43f755970b3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:33.530027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-f945f9d6-f1d3-4a9f-bb58-eec867b8b543 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00fd42c83e1240f2bebfd43f755970b3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:33.542335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-f945f9d6-f1d3-4a9f-bb58-eec867b8b543 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00fd42c83e1240f2bebfd43f755970b3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:33.543464 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-f945f9d6-f1d3-4a9f-bb58-eec867b8b543 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00fd42c83e1240f2bebfd43f755970b3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:33.562056 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-f945f9d6-f1d3-4a9f-bb58-eec867b8b543 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00fd42c83e1240f2bebfd43f755970b3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:33.581223 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1242/2505] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:33 2022] DELETE /identity/v3/users/00fd42c83e1240f2bebfd43f755970b3 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:33.585703 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2d5f1-3415-4993-b17b-b26877379155 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:33.587093 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2d5f1-3415-4993-b17b-b26877379155 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:33.587232 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2d5f1-3415-4993-b17b-b26877379155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:33.623388 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-c4d2d5f1-3415-4993-b17b-b26877379155 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:33.631683 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-c4d2d5f1-3415-4993-b17b-b26877379155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:33.721431 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:33.782561 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1264/2506] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:19:33.786000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945f9d6-f1d3-4a9f-bb58-eec867b8b543 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:33.789428 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-f945f9d6-f1d3-4a9f-bb58-eec867b8b543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:33.863225 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945f9d6-f1d3-4a9f-bb58-eec867b8b543 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:33.867971 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b122cc-045c-459d-92fd-e62b4d56ada5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:33.868738 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b122cc-045c-459d-92fd-e62b4d56ada5 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:33.869018 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b122cc-045c-459d-92fd-e62b4d56ada5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:33.869291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b122cc-045c-459d-92fd-e62b4d56ada5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:33.880100 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b122cc-045c-459d-92fd-e62b4d56ada5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:33.881590 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b122cc-045c-459d-92fd-e62b4d56ada5 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:33.887176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:33.893799 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1243/2507] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:34.205377 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2d5f1-3415-4993-b17b-b26877379155 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:34.207105 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-c4d2d5f1-3415-4993-b17b-b26877379155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:34.227651 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2d5f1-3415-4993-b17b-b26877379155 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:34.233361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe46cba-7b17-4229-9ac8-4b945b3e51cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:34.234264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe46cba-7b17-4229-9ac8-4b945b3e51cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:34.234409 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe46cba-7b17-4229-9ac8-4b945b3e51cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:34.234623 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe46cba-7b17-4229-9ac8-4b945b3e51cd admin admin] PATH_INFO: `/v3/projects/ab3be2f13cb24652b1f6e484d31f335d` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:34.236304 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe46cba-7b17-4229-9ac8-4b945b3e51cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab3be2f13cb24652b1f6e484d31f335d)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:34.238473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe46cba-7b17-4229-9ac8-4b945b3e51cd admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:34.296544 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-cbe46cba-7b17-4229-9ac8-4b945b3e51cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab3be2f13cb24652b1f6e484d31f335d', 'request_id': 'req-cbe46cba-7b17-4229-9ac8-4b945b3e51cd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:34.307363 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-cbe46cba-7b17-4229-9ac8-4b945b3e51cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab3be2f13cb24652b1f6e484d31f335d', 'request_id': 'req-cbe46cba-7b17-4229-9ac8-4b945b3e51cd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:34.316201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-cbe46cba-7b17-4229-9ac8-4b945b3e51cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab3be2f13cb24652b1f6e484d31f335d', 'request_id': 'req-cbe46cba-7b17-4229-9ac8-4b945b3e51cd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:34.330244 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-cbe46cba-7b17-4229-9ac8-4b945b3e51cd admin admin] The token cache is being invalidate because project ab3be2f13cb24652b1f6e484d31f335d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:34.330710 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-cbe46cba-7b17-4229-9ac8-4b945b3e51cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:34.331667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-cbe46cba-7b17-4229-9ac8-4b945b3e51cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:34.333534 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1265/2508] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:34 2022] DELETE /identity/v3/projects/ab3be2f13cb24652b1f6e484d31f335d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:34.338437 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b122cc-045c-459d-92fd-e62b4d56ada5 service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:34.349828 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b122cc-045c-459d-92fd-e62b4d56ada5 service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:34.413895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678d673-3854-4b11-85e0-6f891b4ae76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:34.414597 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678d673-3854-4b11-85e0-6f891b4ae76f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:34.414808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678d673-3854-4b11-85e0-6f891b4ae76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:34.415083 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678d673-3854-4b11-85e0-6f891b4ae76f admin admin] PATH_INFO: `/v3/users/773ff484c9874ca6afed9baa6d604ad6` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:34.416282 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678d673-3854-4b11-85e0-6f891b4ae76f admin admin] RBAC: Authorizing `identity:delete_user(user_id=773ff484c9874ca6afed9baa6d604ad6)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:34.418286 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678d673-3854-4b11-85e0-6f891b4ae76f admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:34.470571 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9678d673-3854-4b11-85e0-6f891b4ae76f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '773ff484c9874ca6afed9baa6d604ad6'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:34.471235 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9678d673-3854-4b11-85e0-6f891b4ae76f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '773ff484c9874ca6afed9baa6d604ad6'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:34.479566 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9678d673-3854-4b11-85e0-6f891b4ae76f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '773ff484c9874ca6afed9baa6d604ad6'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:34.480288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9678d673-3854-4b11-85e0-6f891b4ae76f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '773ff484c9874ca6afed9baa6d604ad6'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:34.497829 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9678d673-3854-4b11-85e0-6f891b4ae76f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '773ff484c9874ca6afed9baa6d604ad6'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:34.515955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1244/2509] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:34 2022] DELETE /identity/v3/users/773ff484c9874ca6afed9baa6d604ad6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:34.856545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe46cba-7b17-4229-9ac8-4b945b3e51cd admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:34.859302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-cbe46cba-7b17-4229-9ac8-4b945b3e51cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:34.890776 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe46cba-7b17-4229-9ac8-4b945b3e51cd admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:35.012413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798259b-60be-4e74-b6e0-79a107aea3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:35.013400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798259b-60be-4e74-b6e0-79a107aea3a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:35.013687 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798259b-60be-4e74-b6e0-79a107aea3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:35.014128 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798259b-60be-4e74-b6e0-79a107aea3a9 admin admin] PATH_INFO: `/v3/projects/42b50839e1164e43b3f05e137bc3cbed` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:35.015317 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798259b-60be-4e74-b6e0-79a107aea3a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42b50839e1164e43b3f05e137bc3cbed)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:35.017281 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798259b-60be-4e74-b6e0-79a107aea3a9 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:35.067655 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6798259b-60be-4e74-b6e0-79a107aea3a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42b50839e1164e43b3f05e137bc3cbed', 'request_id': 'req-6798259b-60be-4e74-b6e0-79a107aea3a9'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:35.075628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6798259b-60be-4e74-b6e0-79a107aea3a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42b50839e1164e43b3f05e137bc3cbed', 'request_id': 'req-6798259b-60be-4e74-b6e0-79a107aea3a9'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:35.080872 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:35.082079 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6798259b-60be-4e74-b6e0-79a107aea3a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42b50839e1164e43b3f05e137bc3cbed', 'request_id': 'req-6798259b-60be-4e74-b6e0-79a107aea3a9'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:35.096412 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6798259b-60be-4e74-b6e0-79a107aea3a9 admin admin] The token cache is being invalidate because project 42b50839e1164e43b3f05e137bc3cbed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:35.096738 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6798259b-60be-4e74-b6e0-79a107aea3a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:35.097423 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6798259b-60be-4e74-b6e0-79a107aea3a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:35.098966 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1266/2510] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:34 2022] DELETE /identity/v3/projects/42b50839e1164e43b3f05e137bc3cbed => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:35.103891 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678d673-3854-4b11-85e0-6f891b4ae76f admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:35.105471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-9678d673-3854-4b11-85e0-6f891b4ae76f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:35.119673 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678d673-3854-4b11-85e0-6f891b4ae76f admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:35.179849 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0ddb3-53e0-48d8-8d43-cc991629efa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:35.180519 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0ddb3-53e0-48d8-8d43-cc991629efa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:35.180852 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0ddb3-53e0-48d8-8d43-cc991629efa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:35.181126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0ddb3-53e0-48d8-8d43-cc991629efa9 admin admin] PATH_INFO: `/v3/users/5ea620fa07ab457fb5302600e2cb05bc` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:35.182314 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0ddb3-53e0-48d8-8d43-cc991629efa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ea620fa07ab457fb5302600e2cb05bc)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:35.184249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0ddb3-53e0-48d8-8d43-cc991629efa9 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:35.210628 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:35.240820 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-75a0ddb3-53e0-48d8-8d43-cc991629efa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ea620fa07ab457fb5302600e2cb05bc'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:35.241487 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-75a0ddb3-53e0-48d8-8d43-cc991629efa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ea620fa07ab457fb5302600e2cb05bc'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:35.254996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-75a0ddb3-53e0-48d8-8d43-cc991629efa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ea620fa07ab457fb5302600e2cb05bc'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:35.255381 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-75a0ddb3-53e0-48d8-8d43-cc991629efa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ea620fa07ab457fb5302600e2cb05bc'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:35.274839 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-75a0ddb3-53e0-48d8-8d43-cc991629efa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ea620fa07ab457fb5302600e2cb05bc'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:35.290462 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1245/2511] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:35 2022] DELETE /identity/v3/users/5ea620fa07ab457fb5302600e2cb05bc => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:35.632098 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798259b-60be-4e74-b6e0-79a107aea3a9 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:35.635100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-6798259b-60be-4e74-b6e0-79a107aea3a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:35.652047 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798259b-60be-4e74-b6e0-79a107aea3a9 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:35.737620 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fc26e-0b71-4b0e-87f8-7f235bcd8d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:35.738277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fc26e-0b71-4b0e-87f8-7f235bcd8d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:35.738560 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fc26e-0b71-4b0e-87f8-7f235bcd8d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:35.739004 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fc26e-0b71-4b0e-87f8-7f235bcd8d3d admin admin] PATH_INFO: `/v3/projects/e9670185ddb9482cbe57627a5f310c65` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:35.740208 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fc26e-0b71-4b0e-87f8-7f235bcd8d3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9670185ddb9482cbe57627a5f310c65)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:35.742227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fc26e-0b71-4b0e-87f8-7f235bcd8d3d admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:35.800094 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ff0fc26e-0b71-4b0e-87f8-7f235bcd8d3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9670185ddb9482cbe57627a5f310c65', 'request_id': 'req-ff0fc26e-0b71-4b0e-87f8-7f235bcd8d3d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:35.809647 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ff0fc26e-0b71-4b0e-87f8-7f235bcd8d3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9670185ddb9482cbe57627a5f310c65', 'request_id': 'req-ff0fc26e-0b71-4b0e-87f8-7f235bcd8d3d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:35.817362 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ff0fc26e-0b71-4b0e-87f8-7f235bcd8d3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9670185ddb9482cbe57627a5f310c65', 'request_id': 'req-ff0fc26e-0b71-4b0e-87f8-7f235bcd8d3d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:35.829995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ff0fc26e-0b71-4b0e-87f8-7f235bcd8d3d admin admin] The token cache is being invalidate because project e9670185ddb9482cbe57627a5f310c65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:35.830329 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ff0fc26e-0b71-4b0e-87f8-7f235bcd8d3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:35.831085 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ff0fc26e-0b71-4b0e-87f8-7f235bcd8d3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:35.833169 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1267/2512] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:35 2022] DELETE /identity/v3/projects/e9670185ddb9482cbe57627a5f310c65 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:35.839347 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb13d6-89c9-419a-a0c4-63a7dc57fb54 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:35.839890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb13d6-89c9-419a-a0c4-63a7dc57fb54 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:35.839890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb13d6-89c9-419a-a0c4-63a7dc57fb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:35.866517 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-06bb13d6-89c9-419a-a0c4-63a7dc57fb54 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:35.866517 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-06bb13d6-89c9-419a-a0c4-63a7dc57fb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:35.930686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1246/2513] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:35.941341 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb5f34-a739-4082-b256-68418d658fe8 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:35.941906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb5f34-a739-4082-b256-68418d658fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:35.941906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb5f34-a739-4082-b256-68418d658fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:35.963510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-63fb5f34-a739-4082-b256-68418d658fe8 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:35.964932 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-63fb5f34-a739-4082-b256-68418d658fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:35.974139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1268/2514] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:35.977291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb13d6-89c9-419a-a0c4-63a7dc57fb54 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:35.987833 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb13d6-89c9-419a-a0c4-63a7dc57fb54 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:35.992495 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc5771-297c-4f2a-9bd8-ec60c653c608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:35.993108 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc5771-297c-4f2a-9bd8-ec60c653c608 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:35.993383 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc5771-297c-4f2a-9bd8-ec60c653c608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:35.993646 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc5771-297c-4f2a-9bd8-ec60c653c608 admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:35.994345 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc5771-297c-4f2a-9bd8-ec60c653c608 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:35.994635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc5771-297c-4f2a-9bd8-ec60c653c608 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:35.996235 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc5771-297c-4f2a-9bd8-ec60c653c608 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:36.019254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1247/2515] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:36.021588 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb5f34-a739-4082-b256-68418d658fe8 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:36.032453 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb5f34-a739-4082-b256-68418d658fe8 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:36.038831 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae3467-94cb-4a10-a747-f08da6c9121f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:36.040250 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae3467-94cb-4a10-a747-f08da6c9121f admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.040565 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae3467-94cb-4a10-a747-f08da6c9121f admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.040841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae3467-94cb-4a10-a747-f08da6c9121f admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.041520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae3467-94cb-4a10-a747-f08da6c9121f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:36.043698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae3467-94cb-4a10-a747-f08da6c9121f admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:36.075502 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1269/2516] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:36 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:36.080031 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc5771-297c-4f2a-9bd8-ec60c653c608 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:36.090283 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc5771-297c-4f2a-9bd8-ec60c653c608 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:36.161800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9966f1d-dd9e-4077-ba71-cb0743b5fe6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:36.162835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9966f1d-dd9e-4077-ba71-cb0743b5fe6e admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.163311 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9966f1d-dd9e-4077-ba71-cb0743b5fe6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.163789 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9966f1d-dd9e-4077-ba71-cb0743b5fe6e admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.165061 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9966f1d-dd9e-4077-ba71-cb0743b5fe6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:36.168917 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9966f1d-dd9e-4077-ba71-cb0743b5fe6e admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:36.200693 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1248/2517] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:36 2022] POST /identity/v3/users => generated 283 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:36.205597 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae3467-94cb-4a10-a747-f08da6c9121f admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:36.218574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae3467-94cb-4a10-a747-f08da6c9121f admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:36.223503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2645259-2572-4b17-a9a2-7044f3a1ba12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:36.224340 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2645259-2572-4b17-a9a2-7044f3a1ba12 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.224791 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2645259-2572-4b17-a9a2-7044f3a1ba12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.225068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2645259-2572-4b17-a9a2-7044f3a1ba12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.225774 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2645259-2572-4b17-a9a2-7044f3a1ba12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:36.227462 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2645259-2572-4b17-a9a2-7044f3a1ba12 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:36.239819 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1270/2518] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:36.244792 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9966f1d-dd9e-4077-ba71-cb0743b5fe6e admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:36.258681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9966f1d-dd9e-4077-ba71-cb0743b5fe6e admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:36.268711 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e638e5-c88f-4cef-a22f-d488d9246a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:36.269470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e638e5-c88f-4cef-a22f-d488d9246a84 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.269630 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e638e5-c88f-4cef-a22f-d488d9246a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.269887 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e638e5-c88f-4cef-a22f-d488d9246a84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.270634 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e638e5-c88f-4cef-a22f-d488d9246a84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:36.272506 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e638e5-c88f-4cef-a22f-d488d9246a84 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:36.293058 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2645259-2572-4b17-a9a2-7044f3a1ba12 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:36.300911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1249/2519] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:36.308414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2645259-2572-4b17-a9a2-7044f3a1ba12 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:36.325596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6709c6-f0df-4b99-85b4-c796266f7d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:36.326507 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6709c6-f0df-4b99-85b4-c796266f7d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.326767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6709c6-f0df-4b99-85b4-c796266f7d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.327149 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6709c6-f0df-4b99-85b4-c796266f7d89 admin admin] PATH_INFO: `/v3/projects/19285acbab524426af55e76c99dd991c/users/e691ed223f1444dda559d96ef00200cd/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.348341 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6709c6-f0df-4b99-85b4-c796266f7d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19285acbab524426af55e76c99dd991c, user_id=e691ed223f1444dda559d96ef00200cd, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:36.350257 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6709c6-f0df-4b99-85b4-c796266f7d89 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:36.373754 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1271/2520] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:36 2022] PUT /identity/v3/projects/19285acbab524426af55e76c99dd991c/users/e691ed223f1444dda559d96ef00200cd/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:36.385539 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c824da-e2e3-417a-b39d-d9023eccde7e None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.385957 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c824da-e2e3-417a-b39d-d9023eccde7e None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.386141 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c824da-e2e3-417a-b39d-d9023eccde7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.396144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:36.435392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-f7c824da-e2e3-417a-b39d-d9023eccde7e None None] MFA Rules not processed for user `e691ed223f1444dda559d96ef00200cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:36.436729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-f7c824da-e2e3-417a-b39d-d9023eccde7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:36.545883 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1250/2521] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:36.554741 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6709c6-f0df-4b99-85b4-c796266f7d89 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:36.590105 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:36.651330 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6709c6-f0df-4b99-85b4-c796266f7d89 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:36.657345 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a93a0a-c320-4340-80ea-ff48d3f3de3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:36.658188 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a93a0a-c320-4340-80ea-ff48d3f3de3b admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.658684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a93a0a-c320-4340-80ea-ff48d3f3de3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.658888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a93a0a-c320-4340-80ea-ff48d3f3de3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.659729 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a93a0a-c320-4340-80ea-ff48d3f3de3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:36.661903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a93a0a-c320-4340-80ea-ff48d3f3de3b admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:36.704090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1272/2522] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:36 2022] POST /identity/v3/projects => generated 367 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:36.708426 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c824da-e2e3-417a-b39d-d9023eccde7e None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:36.721244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c824da-e2e3-417a-b39d-d9023eccde7e None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:36.823178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f71b7-1012-4875-b8d5-b89872a778ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:36.823941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f71b7-1012-4875-b8d5-b89872a778ed admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.824225 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f71b7-1012-4875-b8d5-b89872a778ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.824681 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f71b7-1012-4875-b8d5-b89872a778ed admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.826387 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f71b7-1012-4875-b8d5-b89872a778ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:36.828684 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f71b7-1012-4875-b8d5-b89872a778ed admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:36.862276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1251/2523] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:36 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:36.866868 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a93a0a-c320-4340-80ea-ff48d3f3de3b admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:36.888975 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a93a0a-c320-4340-80ea-ff48d3f3de3b admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:36.904029 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efbc7d3-2f04-4c2f-9a1a-ab56a70791d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:36.906279 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbc7d3-2f04-4c2f-9a1a-ab56a70791d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.907293 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbc7d3-2f04-4c2f-9a1a-ab56a70791d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.908946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbc7d3-2f04-4c2f-9a1a-ab56a70791d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.910563 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efbc7d3-2f04-4c2f-9a1a-ab56a70791d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:36.913903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efbc7d3-2f04-4c2f-9a1a-ab56a70791d5 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:36.934438 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1273/2524] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:19:36.936248 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f71b7-1012-4875-b8d5-b89872a778ed admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:36.951857 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f71b7-1012-4875-b8d5-b89872a778ed admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:36.959883 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b865bdd-8418-469c-9c66-79a1756061f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:36.959883 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b865bdd-8418-469c-9c66-79a1756061f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:36.960285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b865bdd-8418-469c-9c66-79a1756061f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:36.960414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b865bdd-8418-469c-9c66-79a1756061f6 admin admin] PATH_INFO: `/v3/projects/0f35eaf8822c4673b916e90ada0ed613/users/b01220bda59d4783be66260b93d94502/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:36.984603 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b865bdd-8418-469c-9c66-79a1756061f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f35eaf8822c4673b916e90ada0ed613, user_id=b01220bda59d4783be66260b93d94502, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:36.988169 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b865bdd-8418-469c-9c66-79a1756061f6 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:37.012414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1252/2525] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:36 2022] PUT /identity/v3/projects/0f35eaf8822c4673b916e90ada0ed613/users/b01220bda59d4783be66260b93d94502/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:37.017388 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efbc7d3-2f04-4c2f-9a1a-ab56a70791d5 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:37.029841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efbc7d3-2f04-4c2f-9a1a-ab56a70791d5 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:37.089714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e5e5b-4efe-4277-86a5-d811db05ec35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:37.090335 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e5e5b-4efe-4277-86a5-d811db05ec35 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.090621 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e5e5b-4efe-4277-86a5-d811db05ec35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.090896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e5e5b-4efe-4277-86a5-d811db05ec35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.091758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e5e5b-4efe-4277-86a5-d811db05ec35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:37.093476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e5e5b-4efe-4277-86a5-d811db05ec35 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:37.105631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1274/2526] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:37.110389 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b865bdd-8418-469c-9c66-79a1756061f6 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:37.124338 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b865bdd-8418-469c-9c66-79a1756061f6 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:37.130723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc0315-1bfe-4a63-b675-19e6ec5baecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:37.131757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc0315-1bfe-4a63-b675-19e6ec5baecd admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.131938 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc0315-1bfe-4a63-b675-19e6ec5baecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.132234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc0315-1bfe-4a63-b675-19e6ec5baecd admin admin] PATH_INFO: `/v3/projects/0f35eaf8822c4673b916e90ada0ed613/users/b01220bda59d4783be66260b93d94502/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.134698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc0315-1bfe-4a63-b675-19e6ec5baecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f35eaf8822c4673b916e90ada0ed613, user_id=b01220bda59d4783be66260b93d94502, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:37.137621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc0315-1bfe-4a63-b675-19e6ec5baecd admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:37.150498 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1253/2527] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:37 2022] PUT /identity/v3/projects/0f35eaf8822c4673b916e90ada0ed613/users/b01220bda59d4783be66260b93d94502/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:37.162061 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d8124-f8bf-484a-80cc-e09193cd2c01 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.162061 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d8124-f8bf-484a-80cc-e09193cd2c01 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.162061 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d8124-f8bf-484a-80cc-e09193cd2c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.196111 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-f51d8124-f8bf-484a-80cc-e09193cd2c01 None None] MFA Rules not processed for user `b01220bda59d4783be66260b93d94502`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:37.197362 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f51d8124-f8bf-484a-80cc-e09193cd2c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:37.290295 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1275/2528] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:37 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:19:37.303200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc0315-1bfe-4a63-b675-19e6ec5baecd admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:37.304945 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-39fc0315-1bfe-4a63-b675-19e6ec5baecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:37.364991 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:37.370728 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc0315-1bfe-4a63-b675-19e6ec5baecd admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:37.376781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4628c10-63c6-4595-aa6a-b1745ba4c25d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:37.377522 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4628c10-63c6-4595-aa6a-b1745ba4c25d service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.377907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4628c10-63c6-4595-aa6a-b1745ba4c25d service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.378254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4628c10-63c6-4595-aa6a-b1745ba4c25d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.389153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4628c10-63c6-4595-aa6a-b1745ba4c25d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:37.390785 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4628c10-63c6-4595-aa6a-b1745ba4c25d service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:37.401619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1254/2529] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:37 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:37.405486 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6724b-8fa1-47c9-a9e8-02d881c17102 None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.405813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6724b-8fa1-47c9-a9e8-02d881c17102 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.406053 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6724b-8fa1-47c9-a9e8-02d881c17102 None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.407438 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1276/2530] 10.222.0.93 () {54 vars in 921 bytes} [Mon Aug 1 15:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:19:37.414667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89cb99-e5ca-4c91-bcc4-1945aa2381fa None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.415174 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89cb99-e5ca-4c91-bcc4-1945aa2381fa None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.415477 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89cb99-e5ca-4c91-bcc4-1945aa2381fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.438402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-ce89cb99-e5ca-4c91-bcc4-1945aa2381fa None None] MFA Rules not processed for user `3d9be74621d14c94b95ae94947e5abbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:37.439638 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ce89cb99-e5ca-4c91-bcc4-1945aa2381fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:37.505846 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6724b-8fa1-47c9-a9e8-02d881c17102 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:37.538110 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6724b-8fa1-47c9-a9e8-02d881c17102 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:37.538444 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1255/2531] 10.222.0.93 () {58 vars in 1018 bytes} [Mon Aug 1 15:19:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:37.542732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f05bd-7fb2-49c7-9455-f6f2bc1ab0e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:37.543594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f05bd-7fb2-49c7-9455-f6f2bc1ab0e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.544275 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f05bd-7fb2-49c7-9455-f6f2bc1ab0e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.544275 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f05bd-7fb2-49c7-9455-f6f2bc1ab0e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.555259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89cb99-e5ca-4c91-bcc4-1945aa2381fa None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:37.557061 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:37.557971 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ce89cb99-e5ca-4c91-bcc4-1945aa2381fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:37.607620 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754f05bd-7fb2-49c7-9455-f6f2bc1ab0e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:37.609111 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754f05bd-7fb2-49c7-9455-f6f2bc1ab0e9 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:37.618456 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1277/2532] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:37 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:37.619864 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89cb99-e5ca-4c91-bcc4-1945aa2381fa None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:37.624142 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e18c1-ce50-42f1-a5f7-fc1ad7bd37ec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4461a389da5848f5afd9854758707829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:37.624867 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e18c1-ce50-42f1-a5f7-fc1ad7bd37ec service glance] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.625082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e18c1-ce50-42f1-a5f7-fc1ad7bd37ec service glance] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.625360 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e18c1-ce50-42f1-a5f7-fc1ad7bd37ec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.636075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e18c1-ce50-42f1-a5f7-fc1ad7bd37ec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:37.637552 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e18c1-ce50-42f1-a5f7-fc1ad7bd37ec service glance] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:37.648847 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1256/2533] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:19:37.729523 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1794178-c694-4ec7-a1a8-2691fcfd739d None None] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.729807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1794178-c694-4ec7-a1a8-2691fcfd739d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.730078 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1794178-c694-4ec7-a1a8-2691fcfd739d None None] PATH_INFO: `/` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.731612 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1278/2534] 10.222.0.93 () {54 vars in 921 bytes} [Mon Aug 1 15:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:37.737387 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ab1f6-7c4f-4be5-b286-6f83da28c140 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.737750 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ab1f6-7c4f-4be5-b286-6f83da28c140 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.738097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ab1f6-7c4f-4be5-b286-6f83da28c140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.760217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-629ab1f6-7c4f-4be5-b286-6f83da28c140 None None] MFA Rules not processed for user `4eba5f64833b482d81c53721c8442970`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:37.761521 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-629ab1f6-7c4f-4be5-b286-6f83da28c140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:37.822614 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1257/2535] 10.222.0.93 () {58 vars in 1018 bytes} [Mon Aug 1 15:19:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:37.835774 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1794178-c694-4ec7-a1a8-2691fcfd739d None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:37.837838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-c1794178-c694-4ec7-a1a8-2691fcfd739d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:37.852986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1794178-c694-4ec7-a1a8-2691fcfd739d None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:37.859511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f059f73-9a21-4cdd-8055-dfe6593b1b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:37.861013 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f059f73-9a21-4cdd-8055-dfe6593b1b85 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.861420 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f059f73-9a21-4cdd-8055-dfe6593b1b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.861808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f059f73-9a21-4cdd-8055-dfe6593b1b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.872140 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f059f73-9a21-4cdd-8055-dfe6593b1b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:37.873793 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f059f73-9a21-4cdd-8055-dfe6593b1b85 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:37.882754 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1279/2536] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:37.980844 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3194149-a218-4fa4-8d64-2c7489b49ae5 None None] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.981415 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3194149-a218-4fa4-8d64-2c7489b49ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.981415 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3194149-a218-4fa4-8d64-2c7489b49ae5 None None] PATH_INFO: `/` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:37.983670 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1258/2537] 10.222.0.93 () {54 vars in 921 bytes} [Mon Aug 1 15:19:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:37.990017 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ee553-d0c9-420b-8d79-ef0daee259f2 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:37.990384 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ee553-d0c9-420b-8d79-ef0daee259f2 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:37.990733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ee553-d0c9-420b-8d79-ef0daee259f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:38.012267 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-251ee553-d0c9-420b-8d79-ef0daee259f2 None None] MFA Rules not processed for user `3d9be74621d14c94b95ae94947e5abbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:38.013481 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-251ee553-d0c9-420b-8d79-ef0daee259f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:38.024142 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1280/2538] 10.222.0.93 () {58 vars in 1018 bytes} [Mon Aug 1 15:19:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:38.035545 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3194149-a218-4fa4-8d64-2c7489b49ae5 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:38.037314 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e3194149-a218-4fa4-8d64-2c7489b49ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:38.051583 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3194149-a218-4fa4-8d64-2c7489b49ae5 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:38.055268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0842e8d-ee40-4ca8-9d70-155eec664ce6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:38.055889 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0842e8d-ee40-4ca8-9d70-155eec664ce6 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:38.056164 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0842e8d-ee40-4ca8-9d70-155eec664ce6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:38.056454 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0842e8d-ee40-4ca8-9d70-155eec664ce6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:38.068018 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0842e8d-ee40-4ca8-9d70-155eec664ce6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:38.069494 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0842e8d-ee40-4ca8-9d70-155eec664ce6 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:38.092572 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1259/2539] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:39.854022 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675541ce-4c00-4058-b054-bae940e93168 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:39.854605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675541ce-4c00-4058-b054-bae940e93168 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:39.854605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675541ce-4c00-4058-b054-bae940e93168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:39.874673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-675541ce-4c00-4058-b054-bae940e93168 None None] MFA Rules not processed for user `4370f8f474c2446e989d0d8c5b75e87a`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:39.875887 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-675541ce-4c00-4058-b054-bae940e93168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:39.935204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1281/2540] 10.222.0.93 () {58 vars in 1009 bytes} [Mon Aug 1 15:19:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:39.948334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0842e8d-ee40-4ca8-9d70-155eec664ce6 service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:39.949911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e0842e8d-ee40-4ca8-9d70-155eec664ce6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:39.975679 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:40.025268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0842e8d-ee40-4ca8-9d70-155eec664ce6 service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:40.030998 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc33175-8a7c-42e2-b6e7-1784609fc1ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4df37b02152c48509243b73ebbf49d8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:40.031624 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc33175-8a7c-42e2-b6e7-1784609fc1ce service swift] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:40.031900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc33175-8a7c-42e2-b6e7-1784609fc1ce service swift] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:40.032171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc33175-8a7c-42e2-b6e7-1784609fc1ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:40.040816 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc33175-8a7c-42e2-b6e7-1784609fc1ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:40.042405 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc33175-8a7c-42e2-b6e7-1784609fc1ce service swift] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:40.050832 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1260/2541] 10.222.0.93 () {60 vars in 1384 bytes} [Mon Aug 1 15:19:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 15:19:40.604130 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675541ce-4c00-4058-b054-bae940e93168 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:40.615141 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675541ce-4c00-4058-b054-bae940e93168 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:40.619461 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e405436-2834-417e-8b17-811823dc115a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:40.620383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e405436-2834-417e-8b17-811823dc115a service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:40.620860 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e405436-2834-417e-8b17-811823dc115a service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:40.621372 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e405436-2834-417e-8b17-811823dc115a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:40.624224 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5e405436-2834-417e-8b17-811823dc115a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:40.728721 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e405436-2834-417e-8b17-811823dc115a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:40.730927 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e405436-2834-417e-8b17-811823dc115a service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:40.752852 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1282/2542] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:43.444750 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc33175-8a7c-42e2-b6e7-1784609fc1ce service swift] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:43.463196 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc33175-8a7c-42e2-b6e7-1784609fc1ce service swift] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:43.475093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f2b42-8cdd-4eff-95f8-f11e314f2e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:43.475857 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f2b42-8cdd-4eff-95f8-f11e314f2e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:43.476203 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f2b42-8cdd-4eff-95f8-f11e314f2e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:43.476570 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f2b42-8cdd-4eff-95f8-f11e314f2e8a admin admin] PATH_INFO: `/v3/users/e691ed223f1444dda559d96ef00200cd` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:43.478034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f2b42-8cdd-4eff-95f8-f11e314f2e8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e691ed223f1444dda559d96ef00200cd)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:43.480569 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f2b42-8cdd-4eff-95f8-f11e314f2e8a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:43.580773 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9c9f2b42-8cdd-4eff-95f8-f11e314f2e8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e691ed223f1444dda559d96ef00200cd'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:43.581871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9c9f2b42-8cdd-4eff-95f8-f11e314f2e8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e691ed223f1444dda559d96ef00200cd'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:43.594135 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9c9f2b42-8cdd-4eff-95f8-f11e314f2e8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e691ed223f1444dda559d96ef00200cd'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:43.595294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9c9f2b42-8cdd-4eff-95f8-f11e314f2e8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e691ed223f1444dda559d96ef00200cd'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:43.626466 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9c9f2b42-8cdd-4eff-95f8-f11e314f2e8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e691ed223f1444dda559d96ef00200cd'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:43.648103 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1261/2543] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:43 2022] DELETE /identity/v3/users/e691ed223f1444dda559d96ef00200cd => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:43.653079 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d323ca-7c0a-4612-a9ee-29a75735ee1e None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:43.655168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d323ca-7c0a-4612-a9ee-29a75735ee1e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:43.655677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d323ca-7c0a-4612-a9ee-29a75735ee1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:43.684972 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-69d323ca-7c0a-4612-a9ee-29a75735ee1e None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:43.686951 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-69d323ca-7c0a-4612-a9ee-29a75735ee1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:43.776253 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:43.898443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1283/2544] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:43.909469 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f2b42-8cdd-4eff-95f8-f11e314f2e8a admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:43.911833 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-9c9f2b42-8cdd-4eff-95f8-f11e314f2e8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:44.060762 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f2b42-8cdd-4eff-95f8-f11e314f2e8a admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:44.067329 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1dafcc-778e-4067-bcef-7aa25cd87cbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:44.068948 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dafcc-778e-4067-bcef-7aa25cd87cbb service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:44.069476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dafcc-778e-4067-bcef-7aa25cd87cbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:44.070008 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dafcc-778e-4067-bcef-7aa25cd87cbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:44.083277 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1dafcc-778e-4067-bcef-7aa25cd87cbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:44.087093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1dafcc-778e-4067-bcef-7aa25cd87cbb service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:44.104281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1262/2545] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:44.617258 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d323ca-7c0a-4612-a9ee-29a75735ee1e None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:44.618707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-69d323ca-7c0a-4612-a9ee-29a75735ee1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:44.647638 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d323ca-7c0a-4612-a9ee-29a75735ee1e None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:44.686948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f74f2-34d1-4cbe-878a-88145efe1c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:44.688572 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f74f2-34d1-4cbe-878a-88145efe1c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:44.689230 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f74f2-34d1-4cbe-878a-88145efe1c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:44.689779 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f74f2-34d1-4cbe-878a-88145efe1c1b admin admin] PATH_INFO: `/v3/projects/19285acbab524426af55e76c99dd991c` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:44.692488 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f74f2-34d1-4cbe-878a-88145efe1c1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=19285acbab524426af55e76c99dd991c)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:44.696663 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f74f2-34d1-4cbe-878a-88145efe1c1b admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:44.842451 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6c5f74f2-34d1-4cbe-878a-88145efe1c1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19285acbab524426af55e76c99dd991c', 'request_id': 'req-6c5f74f2-34d1-4cbe-878a-88145efe1c1b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:44.858058 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6c5f74f2-34d1-4cbe-878a-88145efe1c1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19285acbab524426af55e76c99dd991c', 'request_id': 'req-6c5f74f2-34d1-4cbe-878a-88145efe1c1b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:44.894349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6c5f74f2-34d1-4cbe-878a-88145efe1c1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19285acbab524426af55e76c99dd991c', 'request_id': 'req-6c5f74f2-34d1-4cbe-878a-88145efe1c1b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:44.935133 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6c5f74f2-34d1-4cbe-878a-88145efe1c1b admin admin] The token cache is being invalidate because project 19285acbab524426af55e76c99dd991c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:44.935861 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6c5f74f2-34d1-4cbe-878a-88145efe1c1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:44.943212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6c5f74f2-34d1-4cbe-878a-88145efe1c1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:44.947076 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1dafcc-778e-4067-bcef-7aa25cd87cbb service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:44.947641 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1284/2546] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:44 2022] DELETE /identity/v3/projects/19285acbab524426af55e76c99dd991c => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:44.965863 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1dafcc-778e-4067-bcef-7aa25cd87cbb service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:45.032334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:45.098587 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f40c51-1199-47e0-8a15-748a7204a144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:45.100200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f40c51-1199-47e0-8a15-748a7204a144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:45.100907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f40c51-1199-47e0-8a15-748a7204a144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:45.101616 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f40c51-1199-47e0-8a15-748a7204a144 admin admin] PATH_INFO: `/v3/users/b01220bda59d4783be66260b93d94502` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:45.104415 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f40c51-1199-47e0-8a15-748a7204a144 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b01220bda59d4783be66260b93d94502)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:45.111649 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f40c51-1199-47e0-8a15-748a7204a144 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:45.257521 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-86f40c51-1199-47e0-8a15-748a7204a144 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b01220bda59d4783be66260b93d94502'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:45.259058 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-86f40c51-1199-47e0-8a15-748a7204a144 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b01220bda59d4783be66260b93d94502'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:45.277246 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-86f40c51-1199-47e0-8a15-748a7204a144 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b01220bda59d4783be66260b93d94502'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:45.277513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-86f40c51-1199-47e0-8a15-748a7204a144 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b01220bda59d4783be66260b93d94502'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:45.314619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-86f40c51-1199-47e0-8a15-748a7204a144 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b01220bda59d4783be66260b93d94502'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:45.369959 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1263/2547] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:44 2022] DELETE /identity/v3/users/b01220bda59d4783be66260b93d94502 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:46.177308 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f74f2-34d1-4cbe-878a-88145efe1c1b admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:46.179017 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-6c5f74f2-34d1-4cbe-878a-88145efe1c1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:46.207210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f74f2-34d1-4cbe-878a-88145efe1c1b admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:46.333846 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c808e8-5fb7-41d2-ba08-f2df46a53cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:46.334719 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c808e8-5fb7-41d2-ba08-f2df46a53cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:46.334887 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c808e8-5fb7-41d2-ba08-f2df46a53cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:46.335261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c808e8-5fb7-41d2-ba08-f2df46a53cb5 admin admin] PATH_INFO: `/v3/projects/0f35eaf8822c4673b916e90ada0ed613` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:46.337513 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c808e8-5fb7-41d2-ba08-f2df46a53cb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f35eaf8822c4673b916e90ada0ed613)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:46.339817 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c808e8-5fb7-41d2-ba08-f2df46a53cb5 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:46.451236 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a4c808e8-5fb7-41d2-ba08-f2df46a53cb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f35eaf8822c4673b916e90ada0ed613', 'request_id': 'req-a4c808e8-5fb7-41d2-ba08-f2df46a53cb5'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:46.473904 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a4c808e8-5fb7-41d2-ba08-f2df46a53cb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f35eaf8822c4673b916e90ada0ed613', 'request_id': 'req-a4c808e8-5fb7-41d2-ba08-f2df46a53cb5'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:46.476216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:46.495771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a4c808e8-5fb7-41d2-ba08-f2df46a53cb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f35eaf8822c4673b916e90ada0ed613', 'request_id': 'req-a4c808e8-5fb7-41d2-ba08-f2df46a53cb5'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:46.526350 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a4c808e8-5fb7-41d2-ba08-f2df46a53cb5 admin admin] The token cache is being invalidate because project 0f35eaf8822c4673b916e90ada0ed613 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:46.527726 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a4c808e8-5fb7-41d2-ba08-f2df46a53cb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:46.529871 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a4c808e8-5fb7-41d2-ba08-f2df46a53cb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:46.537594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1285/2548] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:46 2022] DELETE /identity/v3/projects/0f35eaf8822c4673b916e90ada0ed613 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:46.539631 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d960c8-0512-419f-bb1d-5097292ff537 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:46.540128 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d960c8-0512-419f-bb1d-5097292ff537 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:46.543058 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d960c8-0512-419f-bb1d-5097292ff537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:46.594520 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-c7d960c8-0512-419f-bb1d-5097292ff537 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:46.603832 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c7d960c8-0512-419f-bb1d-5097292ff537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:46.688505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1264/2549] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:46.697195 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f751d-f1ba-47d6-b0c4-c57b8ca1e18a None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:46.698016 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f751d-f1ba-47d6-b0c4-c57b8ca1e18a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:46.698538 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f751d-f1ba-47d6-b0c4-c57b8ca1e18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:46.756558 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-8f9f751d-f1ba-47d6-b0c4-c57b8ca1e18a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:46.757823 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-8f9f751d-f1ba-47d6-b0c4-c57b8ca1e18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:46.779329 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1286/2550] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:46.785003 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d960c8-0512-419f-bb1d-5097292ff537 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:46.806607 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d960c8-0512-419f-bb1d-5097292ff537 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:46.820189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387db20-3305-4ef5-87f9-8a04504c76e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:46.821078 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387db20-3305-4ef5-87f9-8a04504c76e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:46.821458 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387db20-3305-4ef5-87f9-8a04504c76e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:46.821687 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387db20-3305-4ef5-87f9-8a04504c76e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:46.822285 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387db20-3305-4ef5-87f9-8a04504c76e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:46.822675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387db20-3305-4ef5-87f9-8a04504c76e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:46.824544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387db20-3305-4ef5-87f9-8a04504c76e7 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:46.852352 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1265/2551] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:46.857521 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f751d-f1ba-47d6-b0c4-c57b8ca1e18a None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:46.869577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f751d-f1ba-47d6-b0c4-c57b8ca1e18a None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:46.871018 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387db20-3305-4ef5-87f9-8a04504c76e7 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:46.873569 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-4387db20-3305-4ef5-87f9-8a04504c76e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:46.879228 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4785b19-95ec-45d8-8df0-d5166c587fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:46.879960 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4785b19-95ec-45d8-8df0-d5166c587fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:46.880313 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4785b19-95ec-45d8-8df0-d5166c587fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:46.880681 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4785b19-95ec-45d8-8df0-d5166c587fe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:46.882043 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4785b19-95ec-45d8-8df0-d5166c587fe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:46.884492 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4785b19-95ec-45d8-8df0-d5166c587fe6 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:46.920297 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:46.948808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1287/2552] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:46 2022] POST /identity/v3/projects => generated 379 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:46.961031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4785b19-95ec-45d8-8df0-d5166c587fe6 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:46.984722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387db20-3305-4ef5-87f9-8a04504c76e7 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:47.103938 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4785b19-95ec-45d8-8df0-d5166c587fe6 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:47.105582 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8acbf1-23a6-4221-801c-b75d88e81dfd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07e3e25e6a8466fb237dcb836aecda2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:47.106479 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8acbf1-23a6-4221-801c-b75d88e81dfd service placement] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:47.107353 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8acbf1-23a6-4221-801c-b75d88e81dfd service placement] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:47.107730 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8acbf1-23a6-4221-801c-b75d88e81dfd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:47.114104 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4741c92-2de6-46e7-8ea9-89e2ed3e51e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:47.114766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4741c92-2de6-46e7-8ea9-89e2ed3e51e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:47.115098 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4741c92-2de6-46e7-8ea9-89e2ed3e51e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:47.115318 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4741c92-2de6-46e7-8ea9-89e2ed3e51e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:47.116721 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4741c92-2de6-46e7-8ea9-89e2ed3e51e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:47.118051 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4741c92-2de6-46e7-8ea9-89e2ed3e51e4 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:47.127778 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-3d8acbf1-23a6-4221-801c-b75d88e81dfd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:47.159598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1288/2553] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:46 2022] POST /identity/v3/users => generated 290 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:47.160740 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4741c92-2de6-46e7-8ea9-89e2ed3e51e4 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:47.162756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a4741c92-2de6-46e7-8ea9-89e2ed3e51e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:47.166843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8acbf1-23a6-4221-801c-b75d88e81dfd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:47.168802 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8acbf1-23a6-4221-801c-b75d88e81dfd service placement] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:47.184620 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4741c92-2de6-46e7-8ea9-89e2ed3e51e4 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:47.203252 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1266/2554] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:47.206113 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8acbf1-23a6-4221-801c-b75d88e81dfd service placement] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:47.217315 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e510c-4a86-4de6-8922-94c88310a05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:47.224900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e510c-4a86-4de6-8922-94c88310a05e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:47.225409 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e510c-4a86-4de6-8922-94c88310a05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:47.226056 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e510c-4a86-4de6-8922-94c88310a05e admin admin] PATH_INFO: `/v3/users/5a49850f6d8a474d83821f5f49dff06c` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:47.232635 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e510c-4a86-4de6-8922-94c88310a05e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a49850f6d8a474d83821f5f49dff06c)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:47.242792 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e510c-4a86-4de6-8922-94c88310a05e admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:47.256985 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8acbf1-23a6-4221-801c-b75d88e81dfd service placement] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:47.276445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d7c6a-7a2e-4bae-932c-a75d8db1dcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:47.277328 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d7c6a-7a2e-4bae-932c-a75d8db1dcc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:47.277722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d7c6a-7a2e-4bae-932c-a75d8db1dcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:47.278133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d7c6a-7a2e-4bae-932c-a75d8db1dcc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:47.282333 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d7c6a-7a2e-4bae-932c-a75d8db1dcc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:47.285509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d7c6a-7a2e-4bae-932c-a75d8db1dcc7 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:47.300513 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1267/2555] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:47.305802 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d7c6a-7a2e-4bae-932c-a75d8db1dcc7 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:47.320216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d7c6a-7a2e-4bae-932c-a75d8db1dcc7 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:47.327913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1c7f2-8bfd-41c7-bb21-8f0b58bd5b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:47.328780 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1c7f2-8bfd-41c7-bb21-8f0b58bd5b23 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:47.329169 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1c7f2-8bfd-41c7-bb21-8f0b58bd5b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:47.329551 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1c7f2-8bfd-41c7-bb21-8f0b58bd5b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:47.330555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1c7f2-8bfd-41c7-bb21-8f0b58bd5b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:47.332766 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1c7f2-8bfd-41c7-bb21-8f0b58bd5b23 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:47.338976 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f39e510c-4a86-4de6-8922-94c88310a05e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a49850f6d8a474d83821f5f49dff06c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:47.347383 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1268/2556] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:47.356543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1c7f2-8bfd-41c7-bb21-8f0b58bd5b23 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:47.358425 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f39e510c-4a86-4de6-8922-94c88310a05e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a49850f6d8a474d83821f5f49dff06c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:47.379421 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1c7f2-8bfd-41c7-bb21-8f0b58bd5b23 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:47.384933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f39e510c-4a86-4de6-8922-94c88310a05e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a49850f6d8a474d83821f5f49dff06c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:47.389260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15a490-6eda-4410-84f0-686ba8540378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:47.389512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f39e510c-4a86-4de6-8922-94c88310a05e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a49850f6d8a474d83821f5f49dff06c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:47.390092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15a490-6eda-4410-84f0-686ba8540378 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:47.390386 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15a490-6eda-4410-84f0-686ba8540378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:47.392757 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f39e510c-4a86-4de6-8922-94c88310a05e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a49850f6d8a474d83821f5f49dff06c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:47.393295 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15a490-6eda-4410-84f0-686ba8540378 admin admin] PATH_INFO: `/v3/projects/202334a2450e4eb2898702e091deef4b/users/150005ba76344f4ea5cdb8b0703850fb/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:47.432078 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15a490-6eda-4410-84f0-686ba8540378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=202334a2450e4eb2898702e091deef4b, user_id=150005ba76344f4ea5cdb8b0703850fb, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:47.438650 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15a490-6eda-4410-84f0-686ba8540378 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:47.445937 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1289/2557] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:47 2022] DELETE /identity/v3/users/5a49850f6d8a474d83821f5f49dff06c => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:47.491930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1269/2558] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:47 2022] PUT /identity/v3/projects/202334a2450e4eb2898702e091deef4b/users/150005ba76344f4ea5cdb8b0703850fb/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:47.510318 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0c27f-e44b-4621-8f80-e47661a307d7 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:47.510937 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0c27f-e44b-4621-8f80-e47661a307d7 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:47.511765 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0c27f-e44b-4621-8f80-e47661a307d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:47.592101 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-24e0c27f-e44b-4621-8f80-e47661a307d7 None None] MFA Rules not processed for user `150005ba76344f4ea5cdb8b0703850fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:47.593649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-24e0c27f-e44b-4621-8f80-e47661a307d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:47.600068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:DELETE=12,keystone:SELECT=83,keystone:INSERT=5 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:47.801278 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1290/2559] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:47.806438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15a490-6eda-4410-84f0-686ba8540378 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:47.808304 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-be15a490-6eda-4410-84f0-686ba8540378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:47.878908 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0c27f-e44b-4621-8f80-e47661a307d7 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:47.880802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-24e0c27f-e44b-4621-8f80-e47661a307d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:48.001866 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15a490-6eda-4410-84f0-686ba8540378 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:48.015682 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0c27f-e44b-4621-8f80-e47661a307d7 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:48.022103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b6169-5455-4c3e-88f2-652b5213c9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:48.022365 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6ca93-a52a-46ec-aade-19e076cd040d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:48.023271 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b6169-5455-4c3e-88f2-652b5213c9a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:48.023733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b6169-5455-4c3e-88f2-652b5213c9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:48.024244 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b6169-5455-4c3e-88f2-652b5213c9a9 admin admin] PATH_INFO: `/v3/projects/916870664da8419eb5466fd75b15987d` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:48.025052 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6ca93-a52a-46ec-aade-19e076cd040d admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:48.025388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6ca93-a52a-46ec-aade-19e076cd040d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:48.025748 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6ca93-a52a-46ec-aade-19e076cd040d admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:48.026488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6ca93-a52a-46ec-aade-19e076cd040d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:48.026764 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b6169-5455-4c3e-88f2-652b5213c9a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=916870664da8419eb5466fd75b15987d)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:48.030474 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b6169-5455-4c3e-88f2-652b5213c9a9 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:48.034119 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6ca93-a52a-46ec-aade-19e076cd040d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:48.120269 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1270/2560] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:47 2022] POST /identity/v3/projects => generated 379 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:48.127727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6ca93-a52a-46ec-aade-19e076cd040d admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:48.127961 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-de9b6169-5455-4c3e-88f2-652b5213c9a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '916870664da8419eb5466fd75b15987d', 'request_id': 'req-de9b6169-5455-4c3e-88f2-652b5213c9a9'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:48.142233 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:48.164836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-de9b6169-5455-4c3e-88f2-652b5213c9a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '916870664da8419eb5466fd75b15987d', 'request_id': 'req-de9b6169-5455-4c3e-88f2-652b5213c9a9'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:48.231231 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-de9b6169-5455-4c3e-88f2-652b5213c9a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '916870664da8419eb5466fd75b15987d', 'request_id': 'req-de9b6169-5455-4c3e-88f2-652b5213c9a9'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:48.312640 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-de9b6169-5455-4c3e-88f2-652b5213c9a9 admin admin] The token cache is being invalidate because project 916870664da8419eb5466fd75b15987d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:48.313068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-de9b6169-5455-4c3e-88f2-652b5213c9a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:48.315867 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-de9b6169-5455-4c3e-88f2-652b5213c9a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:48.319895 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1291/2561] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:47 2022] DELETE /identity/v3/projects/916870664da8419eb5466fd75b15987d => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:48.325138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b297bf2-686a-4e73-b503-1c39ddf107a2 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:48.325528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b297bf2-686a-4e73-b503-1c39ddf107a2 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:48.325729 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b297bf2-686a-4e73-b503-1c39ddf107a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:48.367527 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-5b297bf2-686a-4e73-b503-1c39ddf107a2 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:48.368937 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5b297bf2-686a-4e73-b503-1c39ddf107a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:48.381082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6ca93-a52a-46ec-aade-19e076cd040d admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:48.390614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1292/2562] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:19:48.396397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ef08e-fcfd-462e-8a76-5ad25b23e66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:48.398299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ef08e-fcfd-462e-8a76-5ad25b23e66d admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:48.398605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fd8b2-599b-4578-a130-3ae192a87e85 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:48.399029 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fd8b2-599b-4578-a130-3ae192a87e85 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:48.399170 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fd8b2-599b-4578-a130-3ae192a87e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:48.399896 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ef08e-fcfd-462e-8a76-5ad25b23e66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:48.400346 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ef08e-fcfd-462e-8a76-5ad25b23e66d admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:48.401214 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ef08e-fcfd-462e-8a76-5ad25b23e66d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:48.404216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ef08e-fcfd-462e-8a76-5ad25b23e66d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:48.435442 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-865fd8b2-599b-4578-a130-3ae192a87e85 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:48.440393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-865fd8b2-599b-4578-a130-3ae192a87e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:48.445006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1271/2563] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:48 2022] POST /identity/v3/users => generated 296 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:48.447449 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ef08e-fcfd-462e-8a76-5ad25b23e66d admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:48.450003 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-123ef08e-fcfd-462e-8a76-5ad25b23e66d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:48.461087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1293/2564] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:19:48.465054 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fd8b2-599b-4578-a130-3ae192a87e85 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:48.484983 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fd8b2-599b-4578-a130-3ae192a87e85 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:48.501515 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0b66b-ecbe-4c43-bd05-ad29bd084562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:48.502267 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0b66b-ecbe-4c43-bd05-ad29bd084562 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:48.502582 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0b66b-ecbe-4c43-bd05-ad29bd084562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:48.502851 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0b66b-ecbe-4c43-bd05-ad29bd084562 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:48.503689 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0b66b-ecbe-4c43-bd05-ad29bd084562 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:48.504016 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0b66b-ecbe-4c43-bd05-ad29bd084562 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:48.504230 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ef08e-fcfd-462e-8a76-5ad25b23e66d admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:48.505941 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0b66b-ecbe-4c43-bd05-ad29bd084562 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:48.531268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c46e8-72c1-4250-acd8-c31267dd48aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:48.533596 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c46e8-72c1-4250-acd8-c31267dd48aa admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:48.535835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c46e8-72c1-4250-acd8-c31267dd48aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:48.536619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c46e8-72c1-4250-acd8-c31267dd48aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:48.541351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c46e8-72c1-4250-acd8-c31267dd48aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:48.546432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1294/2565] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:48.547797 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c46e8-72c1-4250-acd8-c31267dd48aa admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:48.552567 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0b66b-ecbe-4c43-bd05-ad29bd084562 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:48.579307 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1272/2566] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:48.579546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0b66b-ecbe-4c43-bd05-ad29bd084562 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:48.581801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c46e8-72c1-4250-acd8-c31267dd48aa admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:48.587877 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428f193-71da-4d50-871d-eb2f8cb62d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:48.589429 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f193-71da-4d50-871d-eb2f8cb62d8b admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:48.590177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f193-71da-4d50-871d-eb2f8cb62d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:48.591741 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f193-71da-4d50-871d-eb2f8cb62d8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:48.593352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428f193-71da-4d50-871d-eb2f8cb62d8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:48.594911 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c46e8-72c1-4250-acd8-c31267dd48aa admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:48.599031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428f193-71da-4d50-871d-eb2f8cb62d8b admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:48.601097 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878087f-e6be-4d73-9df0-2b15bc654b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:48.602207 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878087f-e6be-4d73-9df0-2b15bc654b62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:48.602683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878087f-e6be-4d73-9df0-2b15bc654b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:48.603446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878087f-e6be-4d73-9df0-2b15bc654b62 admin admin] PATH_INFO: `/v3/projects/9e0c22a4bfad44998bec4f23dd1e55c9/users/e6e456271d474bee9fe96c8613714a1b/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:48.620645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:48.629683 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878087f-e6be-4d73-9df0-2b15bc654b62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e0c22a4bfad44998bec4f23dd1e55c9, user_id=e6e456271d474bee9fe96c8613714a1b, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:48.637418 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878087f-e6be-4d73-9df0-2b15bc654b62 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:48.661474 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1295/2567] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:48 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:19:48.663093 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428f193-71da-4d50-871d-eb2f8cb62d8b admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:48.682926 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1273/2568] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:48 2022] PUT /identity/v3/projects/9e0c22a4bfad44998bec4f23dd1e55c9/users/e6e456271d474bee9fe96c8613714a1b/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:48.685350 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878087f-e6be-4d73-9df0-2b15bc654b62 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:48.813958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428f193-71da-4d50-871d-eb2f8cb62d8b admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:48.867754 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878087f-e6be-4d73-9df0-2b15bc654b62 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:48.875509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8628a83-15c0-41b7-8697-5395c5e9e92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:48.876828 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8628a83-15c0-41b7-8697-5395c5e9e92f admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:48.877414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8628a83-15c0-41b7-8697-5395c5e9e92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:48.877944 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8628a83-15c0-41b7-8697-5395c5e9e92f admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:48.879363 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8628a83-15c0-41b7-8697-5395c5e9e92f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:48.882781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8628a83-15c0-41b7-8697-5395c5e9e92f admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:48.910612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1274/2569] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:48.916750 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8628a83-15c0-41b7-8697-5395c5e9e92f admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:48.935734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62572955-afc6-4d32-86f2-9fe9af53a74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:48.937694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62572955-afc6-4d32-86f2-9fe9af53a74c admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:48.937694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62572955-afc6-4d32-86f2-9fe9af53a74c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:48.938244 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62572955-afc6-4d32-86f2-9fe9af53a74c admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:48.938432 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8628a83-15c0-41b7-8697-5395c5e9e92f admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:48.938925 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62572955-afc6-4d32-86f2-9fe9af53a74c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:48.941294 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62572955-afc6-4d32-86f2-9fe9af53a74c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:48.946823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2f363-c3bc-4e83-94e5-2e5bef8808f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:48.948119 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2f363-c3bc-4e83-94e5-2e5bef8808f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:48.948571 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2f363-c3bc-4e83-94e5-2e5bef8808f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:48.949153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2f363-c3bc-4e83-94e5-2e5bef8808f1 admin admin] PATH_INFO: `/v3/projects/9e0c22a4bfad44998bec4f23dd1e55c9/users/e6e456271d474bee9fe96c8613714a1b/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:48.953128 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2f363-c3bc-4e83-94e5-2e5bef8808f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e0c22a4bfad44998bec4f23dd1e55c9, user_id=e6e456271d474bee9fe96c8613714a1b, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:48.962589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2f363-c3bc-4e83-94e5-2e5bef8808f1 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:48.986502 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1275/2570] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:48 2022] PUT /identity/v3/projects/9e0c22a4bfad44998bec4f23dd1e55c9/users/e6e456271d474bee9fe96c8613714a1b/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:48.993403 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1296/2571] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:48 2022] POST /identity/v3/users => generated 283 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:48.998879 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2f363-c3bc-4e83-94e5-2e5bef8808f1 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:48.999713 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729bb74c-c19b-4a89-9fcd-1f1be8c70ac2 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.000153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729bb74c-c19b-4a89-9fcd-1f1be8c70ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.000752 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729bb74c-c19b-4a89-9fcd-1f1be8c70ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.035978 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-729bb74c-c19b-4a89-9fcd-1f1be8c70ac2 None None] MFA Rules not processed for user `e6e456271d474bee9fe96c8613714a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:49.037333 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-729bb74c-c19b-4a89-9fcd-1f1be8c70ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:49.056354 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:49.080996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2f363-c3bc-4e83-94e5-2e5bef8808f1 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:49.086200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1eb142-ca42-462f-9e5b-022092ca8ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:49.086835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1eb142-ca42-462f-9e5b-022092ca8ecf admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.087364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1eb142-ca42-462f-9e5b-022092ca8ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.087655 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1eb142-ca42-462f-9e5b-022092ca8ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.088362 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1eb142-ca42-462f-9e5b-022092ca8ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:49.090028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1eb142-ca42-462f-9e5b-022092ca8ecf admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:49.102592 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1276/2572] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:49.108311 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1eb142-ca42-462f-9e5b-022092ca8ecf admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:49.120120 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1eb142-ca42-462f-9e5b-022092ca8ecf admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:49.121059 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1297/2573] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:49.125766 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219ffbb-56c2-44d8-8290-1d52576de7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:49.126413 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219ffbb-56c2-44d8-8290-1d52576de7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.127064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219ffbb-56c2-44d8-8290-1d52576de7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.127064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219ffbb-56c2-44d8-8290-1d52576de7b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.127716 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c219ffbb-56c2-44d8-8290-1d52576de7b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:49.129643 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c219ffbb-56c2-44d8-8290-1d52576de7b6 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:49.132204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729bb74c-c19b-4a89-9fcd-1f1be8c70ac2 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:49.133735 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-729bb74c-c19b-4a89-9fcd-1f1be8c70ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:49.141018 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1277/2574] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:19:49.145852 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219ffbb-56c2-44d8-8290-1d52576de7b6 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:49.147619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:49.157040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219ffbb-56c2-44d8-8290-1d52576de7b6 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:49.163014 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ccb7fa-36c4-48f4-a289-b636b179c20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:49.163760 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccb7fa-36c4-48f4-a289-b636b179c20c admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.164074 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccb7fa-36c4-48f4-a289-b636b179c20c admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.164345 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccb7fa-36c4-48f4-a289-b636b179c20c admin admin] PATH_INFO: `/v3/projects/a37e0d867da04ff9906c5650064c70f2/users/21dea351686446dfa140b09c3b451e94/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.182785 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ccb7fa-36c4-48f4-a289-b636b179c20c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a37e0d867da04ff9906c5650064c70f2, user_id=21dea351686446dfa140b09c3b451e94, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:49.185559 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ccb7fa-36c4-48f4-a289-b636b179c20c admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:49.199667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1278/2575] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:49 2022] PUT /identity/v3/projects/a37e0d867da04ff9906c5650064c70f2/users/21dea351686446dfa140b09c3b451e94/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:49.208585 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729bb74c-c19b-4a89-9fcd-1f1be8c70ac2 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:49.212602 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813037c-3e65-4adc-b52e-6e523de13624 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.213148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813037c-3e65-4adc-b52e-6e523de13624 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.213492 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813037c-3e65-4adc-b52e-6e523de13624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.248075 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-5813037c-3e65-4adc-b52e-6e523de13624 None None] MFA Rules not processed for user `21dea351686446dfa140b09c3b451e94`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:49.249373 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-5813037c-3e65-4adc-b52e-6e523de13624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:49.265440 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce34637-0d16-481b-a7ff-267ec995a40d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:49.266169 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce34637-0d16-481b-a7ff-267ec995a40d service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.266334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce34637-0d16-481b-a7ff-267ec995a40d service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.266609 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce34637-0d16-481b-a7ff-267ec995a40d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.320190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1279/2576] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:49 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:19:49.324682 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813037c-3e65-4adc-b52e-6e523de13624 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:49.334348 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce34637-0d16-481b-a7ff-267ec995a40d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:49.336074 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce34637-0d16-481b-a7ff-267ec995a40d service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:49.348609 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1298/2577] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:19:49.431218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813037c-3e65-4adc-b52e-6e523de13624 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:49.436428 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d150f-2753-4c38-882e-a7726dce1267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:49.437095 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d150f-2753-4c38-882e-a7726dce1267 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.437366 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d150f-2753-4c38-882e-a7726dce1267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.437638 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d150f-2753-4c38-882e-a7726dce1267 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.438432 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d150f-2753-4c38-882e-a7726dce1267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:49.440328 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d150f-2753-4c38-882e-a7726dce1267 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:49.476290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1280/2578] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:49 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:49.480763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce34637-0d16-481b-a7ff-267ec995a40d service cinder] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:49.499073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce34637-0d16-481b-a7ff-267ec995a40d service cinder] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:49.623075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d80b8-89e3-4630-bb22-8e0e8cc3be6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:49.624019 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d80b8-89e3-4630-bb22-8e0e8cc3be6b admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.624513 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d80b8-89e3-4630-bb22-8e0e8cc3be6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.625103 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d80b8-89e3-4630-bb22-8e0e8cc3be6b admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.625790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d80b8-89e3-4630-bb22-8e0e8cc3be6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:49.627933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d80b8-89e3-4630-bb22-8e0e8cc3be6b admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:49.661239 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1299/2579] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:49 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:49.665367 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d150f-2753-4c38-882e-a7726dce1267 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:49.676871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d150f-2753-4c38-882e-a7726dce1267 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:49.681921 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5103db2-30b4-4e8d-be57-54d365a93968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:49.682533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5103db2-30b4-4e8d-be57-54d365a93968 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.683221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5103db2-30b4-4e8d-be57-54d365a93968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.683411 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5103db2-30b4-4e8d-be57-54d365a93968 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.684411 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5103db2-30b4-4e8d-be57-54d365a93968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:49.685963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5103db2-30b4-4e8d-be57-54d365a93968 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:49.700000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1281/2580] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:19:49.704999 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d80b8-89e3-4630-bb22-8e0e8cc3be6b admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:49.721299 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d80b8-89e3-4630-bb22-8e0e8cc3be6b admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:49.731200 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2e0ff-52a3-49ca-8f4e-1e6c7866acb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:49.732773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2e0ff-52a3-49ca-8f4e-1e6c7866acb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.733470 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2e0ff-52a3-49ca-8f4e-1e6c7866acb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.733948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2e0ff-52a3-49ca-8f4e-1e6c7866acb1 admin admin] PATH_INFO: `/v3/projects/0677b757dcea46718b5ad67c2571e6d7/users/e45af29476da42b8965b40e1c0d4f071/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.738797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:49.762531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2e0ff-52a3-49ca-8f4e-1e6c7866acb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0677b757dcea46718b5ad67c2571e6d7, user_id=e45af29476da42b8965b40e1c0d4f071, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:49.766567 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2e0ff-52a3-49ca-8f4e-1e6c7866acb1 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:49.783845 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1300/2581] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:49 2022] PUT /identity/v3/projects/0677b757dcea46718b5ad67c2571e6d7/users/e45af29476da42b8965b40e1c0d4f071/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:49.788365 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5103db2-30b4-4e8d-be57-54d365a93968 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:49.795578 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:49.800126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5103db2-30b4-4e8d-be57-54d365a93968 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:49.821172 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2e0ff-52a3-49ca-8f4e-1e6c7866acb1 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:49.885033 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51be0b-89a5-47c8-89a8-3d8cf83940f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:49.885792 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51be0b-89a5-47c8-89a8-3d8cf83940f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.885910 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51be0b-89a5-47c8-89a8-3d8cf83940f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.886090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51be0b-89a5-47c8-89a8-3d8cf83940f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.886796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d51be0b-89a5-47c8-89a8-3d8cf83940f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:49.889106 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d51be0b-89a5-47c8-89a8-3d8cf83940f5 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:49.901174 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1282/2582] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:49.906013 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51be0b-89a5-47c8-89a8-3d8cf83940f5 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:49.917412 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2e0ff-52a3-49ca-8f4e-1e6c7866acb1 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:49.923780 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d3914-e8ff-49d4-ad19-a5f8cfb09a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:49.924885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d3914-e8ff-49d4-ad19-a5f8cfb09a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.925118 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51be0b-89a5-47c8-89a8-3d8cf83940f5 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:49.925278 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d3914-e8ff-49d4-ad19-a5f8cfb09a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.925541 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d3914-e8ff-49d4-ad19-a5f8cfb09a79 admin admin] PATH_INFO: `/v3/users/150005ba76344f4ea5cdb8b0703850fb` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.929538 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d3914-e8ff-49d4-ad19-a5f8cfb09a79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=150005ba76344f4ea5cdb8b0703850fb)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:49.931312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67d502-1f3c-4f03-aed5-f5432ac40939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:49.932420 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d3914-e8ff-49d4-ad19-a5f8cfb09a79 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:49.932672 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67d502-1f3c-4f03-aed5-f5432ac40939 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.932849 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67d502-1f3c-4f03-aed5-f5432ac40939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.932993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67d502-1f3c-4f03-aed5-f5432ac40939 admin admin] PATH_INFO: `/v3/projects/0677b757dcea46718b5ad67c2571e6d7/users/e45af29476da42b8965b40e1c0d4f071/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.936685 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67d502-1f3c-4f03-aed5-f5432ac40939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0677b757dcea46718b5ad67c2571e6d7, user_id=e45af29476da42b8965b40e1c0d4f071, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:49.940765 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67d502-1f3c-4f03-aed5-f5432ac40939 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:49.956776 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1283/2583] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:49 2022] PUT /identity/v3/projects/0677b757dcea46718b5ad67c2571e6d7/users/e45af29476da42b8965b40e1c0d4f071/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:49.969383 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9c7e0-8f04-46f2-a95e-4dc5641523ed None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.969807 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9c7e0-8f04-46f2-a95e-4dc5641523ed None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.970233 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9c7e0-8f04-46f2-a95e-4dc5641523ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.996367 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-246d3914-e8ff-49d4-ad19-a5f8cfb09a79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '150005ba76344f4ea5cdb8b0703850fb'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:50.007171 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-246d3914-e8ff-49d4-ad19-a5f8cfb09a79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '150005ba76344f4ea5cdb8b0703850fb'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:50.009474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-a2f9c7e0-8f04-46f2-a95e-4dc5641523ed None None] MFA Rules not processed for user `e45af29476da42b8965b40e1c0d4f071`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:50.011469 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a2f9c7e0-8f04-46f2-a95e-4dc5641523ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:50.017956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-246d3914-e8ff-49d4-ad19-a5f8cfb09a79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '150005ba76344f4ea5cdb8b0703850fb'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:50.018867 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-246d3914-e8ff-49d4-ad19-a5f8cfb09a79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '150005ba76344f4ea5cdb8b0703850fb'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:50.019900 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-246d3914-e8ff-49d4-ad19-a5f8cfb09a79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '150005ba76344f4ea5cdb8b0703850fb'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:50.043347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1301/2584] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:49 2022] DELETE /identity/v3/users/150005ba76344f4ea5cdb8b0703850fb => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:50.048702 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b59a4-671b-4d11-916b-f911d8edd4ec None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:50.049077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b59a4-671b-4d11-916b-f911d8edd4ec None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:50.049493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b59a4-671b-4d11-916b-f911d8edd4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:50.070454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-e81b59a4-671b-4d11-916b-f911d8edd4ec None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:50.071702 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e81b59a4-671b-4d11-916b-f911d8edd4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:50.110638 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1284/2585] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:49 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:50.124797 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9c7e0-8f04-46f2-a95e-4dc5641523ed None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:50.126239 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a2f9c7e0-8f04-46f2-a95e-4dc5641523ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:50.152558 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1302/2586] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:19:50.163029 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b59a4-671b-4d11-916b-f911d8edd4ec None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:50.164478 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e81b59a4-671b-4d11-916b-f911d8edd4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:50.170056 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:50.190779 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9c7e0-8f04-46f2-a95e-4dc5641523ed None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:50.222552 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:50.235958 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b59a4-671b-4d11-916b-f911d8edd4ec None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:50.242641 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c8503-77e1-4531-9743-540efe5f588e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:50.243754 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c8503-77e1-4531-9743-540efe5f588e service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:50.244199 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c8503-77e1-4531-9743-540efe5f588e service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:50.244745 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c8503-77e1-4531-9743-540efe5f588e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:50.252264 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f0703-3e16-4500-a5cf-2fcca3269ef4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:50.252917 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f0703-3e16-4500-a5cf-2fcca3269ef4 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:50.253198 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f0703-3e16-4500-a5cf-2fcca3269ef4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:50.253477 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f0703-3e16-4500-a5cf-2fcca3269ef4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:50.255104 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c8503-77e1-4531-9743-540efe5f588e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:50.256606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c8503-77e1-4531-9743-540efe5f588e service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:50.267458 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1303/2587] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:50.321170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f0703-3e16-4500-a5cf-2fcca3269ef4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:50.322862 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f0703-3e16-4500-a5cf-2fcca3269ef4 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:50.337088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1285/2588] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:50 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:19:50.609719 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c8503-77e1-4531-9743-540efe5f588e service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:50.617331 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-939c8503-77e1-4531-9743-540efe5f588e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:50.635943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c8503-77e1-4531-9743-540efe5f588e service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:50.644184 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea5439-3ba6-4db7-820e-7d2ea9f31d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:50.645077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea5439-3ba6-4db7-820e-7d2ea9f31d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:50.645414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea5439-3ba6-4db7-820e-7d2ea9f31d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:50.645760 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea5439-3ba6-4db7-820e-7d2ea9f31d79 admin admin] PATH_INFO: `/v3/projects/202334a2450e4eb2898702e091deef4b` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:50.647181 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea5439-3ba6-4db7-820e-7d2ea9f31d79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=202334a2450e4eb2898702e091deef4b)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:50.650532 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea5439-3ba6-4db7-820e-7d2ea9f31d79 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:50.740264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e6ea5439-3ba6-4db7-820e-7d2ea9f31d79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '202334a2450e4eb2898702e091deef4b', 'request_id': 'req-e6ea5439-3ba6-4db7-820e-7d2ea9f31d79'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:50.756230 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e6ea5439-3ba6-4db7-820e-7d2ea9f31d79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '202334a2450e4eb2898702e091deef4b', 'request_id': 'req-e6ea5439-3ba6-4db7-820e-7d2ea9f31d79'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:50.766855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e6ea5439-3ba6-4db7-820e-7d2ea9f31d79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '202334a2450e4eb2898702e091deef4b', 'request_id': 'req-e6ea5439-3ba6-4db7-820e-7d2ea9f31d79'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:50.791837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e6ea5439-3ba6-4db7-820e-7d2ea9f31d79 admin admin] The token cache is being invalidate because project 202334a2450e4eb2898702e091deef4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:50.792445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e6ea5439-3ba6-4db7-820e-7d2ea9f31d79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:50.793623 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e6ea5439-3ba6-4db7-820e-7d2ea9f31d79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:50.798390 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1304/2589] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:50 2022] DELETE /identity/v3/projects/202334a2450e4eb2898702e091deef4b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:50.803792 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f0703-3e16-4500-a5cf-2fcca3269ef4 service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:50.817208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f0703-3e16-4500-a5cf-2fcca3269ef4 service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:50.867737 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea5439-3ba6-4db7-820e-7d2ea9f31d79 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:50.869102 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e6ea5439-3ba6-4db7-820e-7d2ea9f31d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:50.904332 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff7219-ac68-4ce8-acbc-2a499922c734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:50.905653 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff7219-ac68-4ce8-acbc-2a499922c734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:50.905942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff7219-ac68-4ce8-acbc-2a499922c734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:50.906471 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff7219-ac68-4ce8-acbc-2a499922c734 admin admin] PATH_INFO: `/v3/users/e6e456271d474bee9fe96c8613714a1b` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:50.907601 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff7219-ac68-4ce8-acbc-2a499922c734 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6e456271d474bee9fe96c8613714a1b)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:50.909594 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff7219-ac68-4ce8-acbc-2a499922c734 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:50.959812 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea5439-3ba6-4db7-820e-7d2ea9f31d79 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:50.966943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b4fd6-6920-4c66-897d-7c263ef2462a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:50.967608 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b4fd6-6920-4c66-897d-7c263ef2462a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:50.967884 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b4fd6-6920-4c66-897d-7c263ef2462a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:50.968154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b4fd6-6920-4c66-897d-7c263ef2462a admin admin] PATH_INFO: `/v3/users/21dea351686446dfa140b09c3b451e94` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:50.969560 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b4fd6-6920-4c66-897d-7c263ef2462a admin admin] RBAC: Authorizing `identity:delete_user(user_id=21dea351686446dfa140b09c3b451e94)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:50.971430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-83ff7219-ac68-4ce8-acbc-2a499922c734 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6e456271d474bee9fe96c8613714a1b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:50.971589 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b4fd6-6920-4c66-897d-7c263ef2462a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:50.972235 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-83ff7219-ac68-4ce8-acbc-2a499922c734 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6e456271d474bee9fe96c8613714a1b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:50.983035 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-83ff7219-ac68-4ce8-acbc-2a499922c734 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6e456271d474bee9fe96c8613714a1b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:50.983821 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-83ff7219-ac68-4ce8-acbc-2a499922c734 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6e456271d474bee9fe96c8613714a1b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:50.997147 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-83ff7219-ac68-4ce8-acbc-2a499922c734 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6e456271d474bee9fe96c8613714a1b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:51.005428 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:51.011118 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:51.012017 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1286/2590] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:50 2022] DELETE /identity/v3/users/e6e456271d474bee9fe96c8613714a1b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:51.033553 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4e7b4fd6-6920-4c66-897d-7c263ef2462a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21dea351686446dfa140b09c3b451e94'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:51.040793 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4e7b4fd6-6920-4c66-897d-7c263ef2462a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21dea351686446dfa140b09c3b451e94'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:51.052254 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4e7b4fd6-6920-4c66-897d-7c263ef2462a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21dea351686446dfa140b09c3b451e94'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:51.053032 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4e7b4fd6-6920-4c66-897d-7c263ef2462a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21dea351686446dfa140b09c3b451e94'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:51.054057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-4e7b4fd6-6920-4c66-897d-7c263ef2462a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21dea351686446dfa140b09c3b451e94'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:51.089381 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1305/2591] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:50 2022] DELETE /identity/v3/users/21dea351686446dfa140b09c3b451e94 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:51.095783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991338a2-7482-49f3-91e4-185c6eb3df21 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.096156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991338a2-7482-49f3-91e4-185c6eb3df21 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.096510 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991338a2-7482-49f3-91e4-185c6eb3df21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.118735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-991338a2-7482-49f3-91e4-185c6eb3df21 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:51.120238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-991338a2-7482-49f3-91e4-185c6eb3df21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:51.227033 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1287/2592] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:51.234597 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b4fd6-6920-4c66-897d-7c263ef2462a admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.236236 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4e7b4fd6-6920-4c66-897d-7c263ef2462a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:51.301189 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b4fd6-6920-4c66-897d-7c263ef2462a admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:51.304941 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92e21e-202a-4065-814d-e7d01b142723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:51.305556 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92e21e-202a-4065-814d-e7d01b142723 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.305837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92e21e-202a-4065-814d-e7d01b142723 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.306113 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92e21e-202a-4065-814d-e7d01b142723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.318061 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e92e21e-202a-4065-814d-e7d01b142723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:51.319626 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e92e21e-202a-4065-814d-e7d01b142723 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:51.333277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1306/2593] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:51.336403 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991338a2-7482-49f3-91e4-185c6eb3df21 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.338032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-991338a2-7482-49f3-91e4-185c6eb3df21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:51.352802 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991338a2-7482-49f3-91e4-185c6eb3df21 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:51.358993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb44a8-ba65-44da-a178-79178563f87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:51.359808 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb44a8-ba65-44da-a178-79178563f87b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.360083 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb44a8-ba65-44da-a178-79178563f87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.360421 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb44a8-ba65-44da-a178-79178563f87b admin admin] PATH_INFO: `/v3/projects/9e0c22a4bfad44998bec4f23dd1e55c9` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.362415 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb44a8-ba65-44da-a178-79178563f87b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e0c22a4bfad44998bec4f23dd1e55c9)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:51.364809 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb44a8-ba65-44da-a178-79178563f87b admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:51.420746 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3deb44a8-ba65-44da-a178-79178563f87b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e0c22a4bfad44998bec4f23dd1e55c9', 'request_id': 'req-3deb44a8-ba65-44da-a178-79178563f87b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:51.439168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3deb44a8-ba65-44da-a178-79178563f87b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e0c22a4bfad44998bec4f23dd1e55c9', 'request_id': 'req-3deb44a8-ba65-44da-a178-79178563f87b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:51.447613 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3deb44a8-ba65-44da-a178-79178563f87b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e0c22a4bfad44998bec4f23dd1e55c9', 'request_id': 'req-3deb44a8-ba65-44da-a178-79178563f87b'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:51.463155 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3deb44a8-ba65-44da-a178-79178563f87b admin admin] The token cache is being invalidate because project 9e0c22a4bfad44998bec4f23dd1e55c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:51.463439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3deb44a8-ba65-44da-a178-79178563f87b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:51.464115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-3deb44a8-ba65-44da-a178-79178563f87b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:51.465834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1288/2594] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:51 2022] DELETE /identity/v3/projects/9e0c22a4bfad44998bec4f23dd1e55c9 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:51.470731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75acf04-b795-454c-957e-edf6e420200d None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.471143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75acf04-b795-454c-957e-edf6e420200d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.471516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75acf04-b795-454c-957e-edf6e420200d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.493598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-b75acf04-b795-454c-957e-edf6e420200d None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:51.494875 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-b75acf04-b795-454c-957e-edf6e420200d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:51.565995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1307/2595] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:51.574149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3079317-828d-4979-8c05-3b3704187888 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.574429 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3079317-828d-4979-8c05-3b3704187888 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.574708 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3079317-828d-4979-8c05-3b3704187888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.597907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-d3079317-828d-4979-8c05-3b3704187888 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:51.599170 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d3079317-828d-4979-8c05-3b3704187888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:51.608407 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1289/2596] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:19:51.612832 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75acf04-b795-454c-957e-edf6e420200d None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.626006 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75acf04-b795-454c-957e-edf6e420200d None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:51.633873 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c18487-2242-421e-9ae3-cb5c29e987e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:51.634532 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c18487-2242-421e-9ae3-cb5c29e987e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.634865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c18487-2242-421e-9ae3-cb5c29e987e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.635080 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c18487-2242-421e-9ae3-cb5c29e987e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.635779 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c18487-2242-421e-9ae3-cb5c29e987e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:51.636081 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c18487-2242-421e-9ae3-cb5c29e987e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:51.637959 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c18487-2242-421e-9ae3-cb5c29e987e8 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:51.647633 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3079317-828d-4979-8c05-3b3704187888 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.650165 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d3079317-828d-4979-8c05-3b3704187888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:51.652021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1308/2597] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:51.657040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c18487-2242-421e-9ae3-cb5c29e987e8 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.667902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c18487-2242-421e-9ae3-cb5c29e987e8 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:51.672665 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3079317-828d-4979-8c05-3b3704187888 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:51.678227 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a437466-0f06-491b-bd27-e546633a7563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:51.678987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a437466-0f06-491b-bd27-e546633a7563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.679125 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9cbaa-a617-45c5-b3fb-b015b378f166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:51.679382 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a437466-0f06-491b-bd27-e546633a7563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.679812 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9cbaa-a617-45c5-b3fb-b015b378f166 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.679950 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a437466-0f06-491b-bd27-e546633a7563 admin admin] PATH_INFO: `/v3/projects/a37e0d867da04ff9906c5650064c70f2` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.680063 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9cbaa-a617-45c5-b3fb-b015b378f166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.680299 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9cbaa-a617-45c5-b3fb-b015b378f166 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.681053 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9cbaa-a617-45c5-b3fb-b015b378f166 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:51.682111 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a437466-0f06-491b-bd27-e546633a7563 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a37e0d867da04ff9906c5650064c70f2)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:51.682980 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9cbaa-a617-45c5-b3fb-b015b378f166 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:51.684526 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a437466-0f06-491b-bd27-e546633a7563 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:51.719057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1309/2598] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:51 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:51.724833 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9cbaa-a617-45c5-b3fb-b015b378f166 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.749222 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9a437466-0f06-491b-bd27-e546633a7563 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a37e0d867da04ff9906c5650064c70f2', 'request_id': 'req-9a437466-0f06-491b-bd27-e546633a7563'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:51.749944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:51.773052 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9a437466-0f06-491b-bd27-e546633a7563 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a37e0d867da04ff9906c5650064c70f2', 'request_id': 'req-9a437466-0f06-491b-bd27-e546633a7563'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:51.798441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9a437466-0f06-491b-bd27-e546633a7563 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a37e0d867da04ff9906c5650064c70f2', 'request_id': 'req-9a437466-0f06-491b-bd27-e546633a7563'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:51.812936 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:51.818774 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9a437466-0f06-491b-bd27-e546633a7563 admin admin] The token cache is being invalidate because project a37e0d867da04ff9906c5650064c70f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:51.821039 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9a437466-0f06-491b-bd27-e546633a7563 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:51.821838 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-9a437466-0f06-491b-bd27-e546633a7563 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:51.823872 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1290/2599] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:51 2022] DELETE /identity/v3/projects/a37e0d867da04ff9906c5650064c70f2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:51.829201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a437466-0f06-491b-bd27-e546633a7563 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.831527 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-9a437466-0f06-491b-bd27-e546633a7563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:51.842687 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9cbaa-a617-45c5-b3fb-b015b378f166 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:51.947675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a437466-0f06-491b-bd27-e546633a7563 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:51.954273 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d345f3a-8541-4682-abba-d37d6ed32afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:51.955292 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d345f3a-8541-4682-abba-d37d6ed32afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.955578 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d345f3a-8541-4682-abba-d37d6ed32afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.955853 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d345f3a-8541-4682-abba-d37d6ed32afd admin admin] PATH_INFO: `/v3/users/e45af29476da42b8965b40e1c0d4f071` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.957338 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d345f3a-8541-4682-abba-d37d6ed32afd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e45af29476da42b8965b40e1c0d4f071)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:51.960807 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d345f3a-8541-4682-abba-d37d6ed32afd admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:51.972270 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef93c3b-b91b-4813-a7f3-95567401aae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:51.972943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef93c3b-b91b-4813-a7f3-95567401aae6 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.973226 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef93c3b-b91b-4813-a7f3-95567401aae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.973503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef93c3b-b91b-4813-a7f3-95567401aae6 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.974180 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef93c3b-b91b-4813-a7f3-95567401aae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:51.976143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef93c3b-b91b-4813-a7f3-95567401aae6 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:52.033673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1310/2600] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:51 2022] POST /identity/v3/users => generated 290 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:19:52.037263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6d345f3a-8541-4682-abba-d37d6ed32afd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e45af29476da42b8965b40e1c0d4f071'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:52.038338 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6d345f3a-8541-4682-abba-d37d6ed32afd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e45af29476da42b8965b40e1c0d4f071'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:52.038504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef93c3b-b91b-4813-a7f3-95567401aae6 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:52.051999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6d345f3a-8541-4682-abba-d37d6ed32afd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e45af29476da42b8965b40e1c0d4f071'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:52.052246 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0ef93c3b-b91b-4813-a7f3-95567401aae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:52.053046 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6d345f3a-8541-4682-abba-d37d6ed32afd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e45af29476da42b8965b40e1c0d4f071'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:52.071323 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6d345f3a-8541-4682-abba-d37d6ed32afd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e45af29476da42b8965b40e1c0d4f071'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:52.092753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1291/2601] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:51 2022] DELETE /identity/v3/users/e45af29476da42b8965b40e1c0d4f071 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:52.102719 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef93c3b-b91b-4813-a7f3-95567401aae6 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:52.225855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da2dca-d869-4856-a184-b1d533ea3e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:52.226842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da2dca-d869-4856-a184-b1d533ea3e5c admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.227267 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da2dca-d869-4856-a184-b1d533ea3e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.227835 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da2dca-d869-4856-a184-b1d533ea3e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.228995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da2dca-d869-4856-a184-b1d533ea3e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:52.233332 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da2dca-d869-4856-a184-b1d533ea3e5c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:52.245890 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1311/2602] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:52.253659 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d345f3a-8541-4682-abba-d37d6ed32afd admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:52.260771 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-6d345f3a-8541-4682-abba-d37d6ed32afd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:52.283073 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d345f3a-8541-4682-abba-d37d6ed32afd admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:52.289803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e56fb-d7f7-4871-b9ee-d370aa6434d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:52.291305 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e56fb-d7f7-4871-b9ee-d370aa6434d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.291305 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e56fb-d7f7-4871-b9ee-d370aa6434d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.291305 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e56fb-d7f7-4871-b9ee-d370aa6434d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.291943 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e56fb-d7f7-4871-b9ee-d370aa6434d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:52.293719 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e56fb-d7f7-4871-b9ee-d370aa6434d9 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:52.310589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1292/2603] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:52.312630 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da2dca-d869-4856-a184-b1d533ea3e5c admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:52.323885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da2dca-d869-4856-a184-b1d533ea3e5c admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:52.337700 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5b3c1-5735-4be2-8487-c49de15cc7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:52.338828 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5b3c1-5735-4be2-8487-c49de15cc7d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.339219 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5b3c1-5735-4be2-8487-c49de15cc7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.340122 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5b3c1-5735-4be2-8487-c49de15cc7d6 admin admin] PATH_INFO: `/v3/projects/a73675eec55e4819b1a742858fd66d90/users/5b249ec5ff4a4dbf8362d5974e97e19e/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.366488 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5b3c1-5735-4be2-8487-c49de15cc7d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a73675eec55e4819b1a742858fd66d90, user_id=5b249ec5ff4a4dbf8362d5974e97e19e, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:52.369657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5b3c1-5735-4be2-8487-c49de15cc7d6 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:52.388811 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1312/2604] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:52 2022] PUT /identity/v3/projects/a73675eec55e4819b1a742858fd66d90/users/5b249ec5ff4a4dbf8362d5974e97e19e/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:52.399362 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e870ac-4212-4b71-93da-b407a881600c None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.399657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e870ac-4212-4b71-93da-b407a881600c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.399943 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e870ac-4212-4b71-93da-b407a881600c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.439935 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-d5e870ac-4212-4b71-93da-b407a881600c None None] MFA Rules not processed for user `5b249ec5ff4a4dbf8362d5974e97e19e`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:52.442026 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d5e870ac-4212-4b71-93da-b407a881600c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:52.525898 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5b3c1-5735-4be2-8487-c49de15cc7d6 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:52.530725 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4ec5b3c1-5735-4be2-8487-c49de15cc7d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:52.550913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:52.564059 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1293/2605] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:52.569977 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e870ac-4212-4b71-93da-b407a881600c None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:52.612755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:52.671386 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e870ac-4212-4b71-93da-b407a881600c None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:52.672772 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5b3c1-5735-4be2-8487-c49de15cc7d6 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:52.676523 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a22a6-f38e-42c1-93d3-2cb7ad4983b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:52.677270 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a22a6-f38e-42c1-93d3-2cb7ad4983b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.677572 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804855cc-36cf-49dc-9f09-eff0eac1bca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:52.677736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a22a6-f38e-42c1-93d3-2cb7ad4983b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.678117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a22a6-f38e-42c1-93d3-2cb7ad4983b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.678248 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804855cc-36cf-49dc-9f09-eff0eac1bca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.678659 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804855cc-36cf-49dc-9f09-eff0eac1bca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.678940 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804855cc-36cf-49dc-9f09-eff0eac1bca9 admin admin] PATH_INFO: `/v3/projects/0677b757dcea46718b5ad67c2571e6d7` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.679150 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a22a6-f38e-42c1-93d3-2cb7ad4983b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:52.680130 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804855cc-36cf-49dc-9f09-eff0eac1bca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0677b757dcea46718b5ad67c2571e6d7)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:52.681308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a22a6-f38e-42c1-93d3-2cb7ad4983b1 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:52.682056 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804855cc-36cf-49dc-9f09-eff0eac1bca9 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:52.718243 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1294/2606] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:52 2022] POST /identity/v3/projects => generated 381 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:52.721890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a22a6-f38e-42c1-93d3-2cb7ad4983b1 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:52.745494 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-804855cc-36cf-49dc-9f09-eff0eac1bca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0677b757dcea46718b5ad67c2571e6d7', 'request_id': 'req-804855cc-36cf-49dc-9f09-eff0eac1bca9'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:52.757766 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-804855cc-36cf-49dc-9f09-eff0eac1bca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0677b757dcea46718b5ad67c2571e6d7', 'request_id': 'req-804855cc-36cf-49dc-9f09-eff0eac1bca9'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:52.765077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-804855cc-36cf-49dc-9f09-eff0eac1bca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0677b757dcea46718b5ad67c2571e6d7', 'request_id': 'req-804855cc-36cf-49dc-9f09-eff0eac1bca9'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:52.792257 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-804855cc-36cf-49dc-9f09-eff0eac1bca9 admin admin] The token cache is being invalidate because project 0677b757dcea46718b5ad67c2571e6d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:52.792891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-804855cc-36cf-49dc-9f09-eff0eac1bca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:52.794118 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-804855cc-36cf-49dc-9f09-eff0eac1bca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:52.798164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1313/2607] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:52 2022] DELETE /identity/v3/projects/0677b757dcea46718b5ad67c2571e6d7 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:52.802763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00739cef-fb25-48f4-a404-952b5f47639d None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.803048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00739cef-fb25-48f4-a404-952b5f47639d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.803335 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00739cef-fb25-48f4-a404-952b5f47639d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.815690 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a22a6-f38e-42c1-93d3-2cb7ad4983b1 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:52.826727 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-00739cef-fb25-48f4-a404-952b5f47639d None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:52.827980 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-00739cef-fb25-48f4-a404-952b5f47639d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:52.899457 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3b879-ea6b-40f4-8323-3ddba4e57790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:52.902221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3b879-ea6b-40f4-8323-3ddba4e57790 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.902221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3b879-ea6b-40f4-8323-3ddba4e57790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.902797 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3b879-ea6b-40f4-8323-3ddba4e57790 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.904801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc3b879-ea6b-40f4-8323-3ddba4e57790 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:52.909244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc3b879-ea6b-40f4-8323-3ddba4e57790 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:52.924437 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1314/2608] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:19:52.925963 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d44db9-eb27-4d1b-8a6e-b0e7850ed5e0 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.926253 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d44db9-eb27-4d1b-8a6e-b0e7850ed5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.926530 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d44db9-eb27-4d1b-8a6e-b0e7850ed5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.952122 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1295/2609] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:52 2022] POST /identity/v3/users => generated 297 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:52.959635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3b879-ea6b-40f4-8323-3ddba4e57790 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:52.962817 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-14d44db9-eb27-4d1b-8a6e-b0e7850ed5e0 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:52.963067 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-bdc3b879-ea6b-40f4-8323-3ddba4e57790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:52.965351 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-14d44db9-eb27-4d1b-8a6e-b0e7850ed5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:52.982446 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3b879-ea6b-40f4-8323-3ddba4e57790 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:52.989459 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa8488-6e49-4c03-bfa6-c5b92796a489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:52.991966 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa8488-6e49-4c03-bfa6-c5b92796a489 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.992254 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa8488-6e49-4c03-bfa6-c5b92796a489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.992675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa8488-6e49-4c03-bfa6-c5b92796a489 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.993560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa8488-6e49-4c03-bfa6-c5b92796a489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:52.995797 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa8488-6e49-4c03-bfa6-c5b92796a489 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:53.001920 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1315/2610] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:53.008283 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d44db9-eb27-4d1b-8a6e-b0e7850ed5e0 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:53.012523 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1296/2611] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:53.015525 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa8488-6e49-4c03-bfa6-c5b92796a489 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:53.024298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d44db9-eb27-4d1b-8a6e-b0e7850ed5e0 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:53.030156 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa8488-6e49-4c03-bfa6-c5b92796a489 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:53.032848 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b50ed7-7f55-4163-ae9a-0ce0a1d95657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:53.033955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b50ed7-7f55-4163-ae9a-0ce0a1d95657 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.034482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b50ed7-7f55-4163-ae9a-0ce0a1d95657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.034764 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b50ed7-7f55-4163-ae9a-0ce0a1d95657 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.035522 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b50ed7-7f55-4163-ae9a-0ce0a1d95657 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:53.037408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b50ed7-7f55-4163-ae9a-0ce0a1d95657 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:53.038380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d09416-f2da-4c81-a971-509d0325f514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:53.039396 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d09416-f2da-4c81-a971-509d0325f514 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.039838 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d09416-f2da-4c81-a971-509d0325f514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.040424 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d09416-f2da-4c81-a971-509d0325f514 admin admin] PATH_INFO: `/v3/projects/4002092a626241ee846ed7e46284a246/users/0d40fb09bdd24837b412b0c7ea59c93c/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.041416 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b50ed7-7f55-4163-ae9a-0ce0a1d95657 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:53.045361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:53.089527 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1316/2612] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:53.094065 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b50ed7-7f55-4163-ae9a-0ce0a1d95657 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:53.096074 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d09416-f2da-4c81-a971-509d0325f514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4002092a626241ee846ed7e46284a246, user_id=0d40fb09bdd24837b412b0c7ea59c93c, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:53.103456 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d09416-f2da-4c81-a971-509d0325f514 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:53.130352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b50ed7-7f55-4163-ae9a-0ce0a1d95657 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:53.136098 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1297/2613] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:53 2022] PUT /identity/v3/projects/4002092a626241ee846ed7e46284a246/users/0d40fb09bdd24837b412b0c7ea59c93c/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:53.141631 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d09416-f2da-4c81-a971-509d0325f514 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:53.198524 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:53.245385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7ada5-0e61-412c-8b66-30c1c5844986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:53.246343 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7ada5-0e61-412c-8b66-30c1c5844986 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.246980 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7ada5-0e61-412c-8b66-30c1c5844986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.248123 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7ada5-0e61-412c-8b66-30c1c5844986 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.249012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7ada5-0e61-412c-8b66-30c1c5844986 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:53.254956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7ada5-0e61-412c-8b66-30c1c5844986 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:53.271330 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d09416-f2da-4c81-a971-509d0325f514 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:53.277546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59954afd-0347-4e45-88f3-94faf1a95c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:53.278562 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59954afd-0347-4e45-88f3-94faf1a95c55 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.279205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59954afd-0347-4e45-88f3-94faf1a95c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.279663 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59954afd-0347-4e45-88f3-94faf1a95c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.280622 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59954afd-0347-4e45-88f3-94faf1a95c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:53.282764 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59954afd-0347-4e45-88f3-94faf1a95c55 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:53.301125 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1298/2614] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:53.305567 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59954afd-0347-4e45-88f3-94faf1a95c55 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:53.321604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59954afd-0347-4e45-88f3-94faf1a95c55 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:53.335024 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07f24b-d215-426d-8433-62d0132cd424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:53.335024 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07f24b-d215-426d-8433-62d0132cd424 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.335024 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07f24b-d215-426d-8433-62d0132cd424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.335024 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07f24b-d215-426d-8433-62d0132cd424 admin admin] PATH_INFO: `/v3/projects/4002092a626241ee846ed7e46284a246/users/0d40fb09bdd24837b412b0c7ea59c93c/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.335740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07f24b-d215-426d-8433-62d0132cd424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4002092a626241ee846ed7e46284a246, user_id=0d40fb09bdd24837b412b0c7ea59c93c, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:53.337746 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07f24b-d215-426d-8433-62d0132cd424 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:53.356269 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1299/2615] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:53 2022] PUT /identity/v3/projects/4002092a626241ee846ed7e46284a246/users/0d40fb09bdd24837b412b0c7ea59c93c/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:53.368268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d282566-cf78-4dfe-a85d-3f339bb979ca None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.368709 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d282566-cf78-4dfe-a85d-3f339bb979ca None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.368987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d282566-cf78-4dfe-a85d-3f339bb979ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.379272 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1317/2616] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:53 2022] POST /identity/v3/projects => generated 389 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:53.383779 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7ada5-0e61-412c-8b66-30c1c5844986 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:53.447072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-2d282566-cf78-4dfe-a85d-3f339bb979ca None None] MFA Rules not processed for user `0d40fb09bdd24837b412b0c7ea59c93c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:53.448529 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-2d282566-cf78-4dfe-a85d-3f339bb979ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:53.471720 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7ada5-0e61-412c-8b66-30c1c5844986 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:53.478255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c423092-dac2-488c-9f2f-169d5bb63ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:53.479043 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c423092-dac2-488c-9f2f-169d5bb63ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.479321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c423092-dac2-488c-9f2f-169d5bb63ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.479593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c423092-dac2-488c-9f2f-169d5bb63ff7 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.480277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c423092-dac2-488c-9f2f-169d5bb63ff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:53.483405 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c423092-dac2-488c-9f2f-169d5bb63ff7 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:53.547521 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1318/2617] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:53 2022] POST /identity/v3/users => generated 295 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:53.552747 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c423092-dac2-488c-9f2f-169d5bb63ff7 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:53.572730 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c423092-dac2-488c-9f2f-169d5bb63ff7 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:53.581624 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66174453-88a7-4aa3-bd42-38feea04d027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:53.590656 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66174453-88a7-4aa3-bd42-38feea04d027 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.591172 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66174453-88a7-4aa3-bd42-38feea04d027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.591552 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66174453-88a7-4aa3-bd42-38feea04d027 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.592385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66174453-88a7-4aa3-bd42-38feea04d027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:53.594159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66174453-88a7-4aa3-bd42-38feea04d027 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:53.601635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1300/2618] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:53 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:19:53.609830 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1319/2619] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:53.616868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d282566-cf78-4dfe-a85d-3f339bb979ca None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:53.622748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66174453-88a7-4aa3-bd42-38feea04d027 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:53.626261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-66174453-88a7-4aa3-bd42-38feea04d027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:53.629721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d282566-cf78-4dfe-a85d-3f339bb979ca None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:53.637620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203ae18-19ab-431d-8b36-6d427a2e671d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:53.639103 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203ae18-19ab-431d-8b36-6d427a2e671d admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.639570 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203ae18-19ab-431d-8b36-6d427a2e671d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.640057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203ae18-19ab-431d-8b36-6d427a2e671d admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.640937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203ae18-19ab-431d-8b36-6d427a2e671d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:53.643701 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203ae18-19ab-431d-8b36-6d427a2e671d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:53.658776 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1301/2620] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:53.663938 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203ae18-19ab-431d-8b36-6d427a2e671d admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:53.675983 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203ae18-19ab-431d-8b36-6d427a2e671d admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:53.684516 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2dbb8-170a-4dc7-a565-f2ddc458033d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:53.685908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2dbb8-170a-4dc7-a565-f2ddc458033d admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.686189 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2dbb8-170a-4dc7-a565-f2ddc458033d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.686476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2dbb8-170a-4dc7-a565-f2ddc458033d admin admin] PATH_INFO: `/v3/projects/4e57b8dc36d1451b96a1bf657bbaa8c8/users/afc747b842bf4046a5bda7f6a1ef9caf/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.709405 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d2dbb8-170a-4dc7-a565-f2ddc458033d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e57b8dc36d1451b96a1bf657bbaa8c8, user_id=afc747b842bf4046a5bda7f6a1ef9caf, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:53.710895 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66174453-88a7-4aa3-bd42-38feea04d027 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:53.712725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d2dbb8-170a-4dc7-a565-f2ddc458033d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:53.714870 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24829c5-1ab7-4e7d-b252-05264f0a778e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:53.715558 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24829c5-1ab7-4e7d-b252-05264f0a778e service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.715784 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24829c5-1ab7-4e7d-b252-05264f0a778e service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.716137 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24829c5-1ab7-4e7d-b252-05264f0a778e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.727843 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24829c5-1ab7-4e7d-b252-05264f0a778e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:53.729621 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24829c5-1ab7-4e7d-b252-05264f0a778e service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:53.732315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1302/2621] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:53 2022] PUT /identity/v3/projects/4e57b8dc36d1451b96a1bf657bbaa8c8/users/afc747b842bf4046a5bda7f6a1ef9caf/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:53.741281 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1320/2622] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:53.755216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de878f-79bf-43d9-96ba-344c35c483c2 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.755879 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de878f-79bf-43d9-96ba-344c35c483c2 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.758304 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de878f-79bf-43d9-96ba-344c35c483c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.821874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-e2de878f-79bf-43d9-96ba-344c35c483c2 None None] MFA Rules not processed for user `afc747b842bf4046a5bda7f6a1ef9caf`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:53.823392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e2de878f-79bf-43d9-96ba-344c35c483c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:53.829488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:53.876340 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24829c5-1ab7-4e7d-b252-05264f0a778e service cinder] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:53.901550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24829c5-1ab7-4e7d-b252-05264f0a778e service cinder] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:53.961718 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:53.976158 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91a82d-dacb-4d2a-bcaa-4a456de7144c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:53.977140 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91a82d-dacb-4d2a-bcaa-4a456de7144c service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.977570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91a82d-dacb-4d2a-bcaa-4a456de7144c service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.978273 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91a82d-dacb-4d2a-bcaa-4a456de7144c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.980561 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4b91a82d-dacb-4d2a-bcaa-4a456de7144c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:54.017245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1303/2623] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:54.024289 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de878f-79bf-43d9-96ba-344c35c483c2 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:54.069768 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91a82d-dacb-4d2a-bcaa-4a456de7144c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:54.073041 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91a82d-dacb-4d2a-bcaa-4a456de7144c service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:54.093054 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1321/2624] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:54.203748 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de878f-79bf-43d9-96ba-344c35c483c2 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:54.227208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2dbfcf-3847-4573-90e7-0459022096db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:54.228315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dbfcf-3847-4573-90e7-0459022096db admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:54.234292 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dbfcf-3847-4573-90e7-0459022096db admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:54.234292 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dbfcf-3847-4573-90e7-0459022096db admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:54.235106 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2dbfcf-3847-4573-90e7-0459022096db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:54.238068 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2dbfcf-3847-4573-90e7-0459022096db admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:54.304507 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1304/2625] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:54 2022] POST /identity/v3/projects => generated 387 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:19:54.309824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91a82d-dacb-4d2a-bcaa-4a456de7144c service cinder] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:54.326982 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91a82d-dacb-4d2a-bcaa-4a456de7144c service cinder] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:54.414376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b5a68-9c2e-4d98-828e-dff190331d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:54.415193 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b5a68-9c2e-4d98-828e-dff190331d6f admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:54.415477 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b5a68-9c2e-4d98-828e-dff190331d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:54.415748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b5a68-9c2e-4d98-828e-dff190331d6f admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:54.416772 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b5a68-9c2e-4d98-828e-dff190331d6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:54.419692 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b5a68-9c2e-4d98-828e-dff190331d6f admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:54.446177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1322/2626] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:54 2022] POST /identity/v3/users => generated 300 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:54.451213 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2dbfcf-3847-4573-90e7-0459022096db admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:54.470544 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2dbfcf-3847-4573-90e7-0459022096db admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:54.478555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e5e19e-fa09-4aa6-a367-464b48939cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:54.479315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5e19e-fa09-4aa6-a367-464b48939cbc admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:54.479667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5e19e-fa09-4aa6-a367-464b48939cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:54.480041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5e19e-fa09-4aa6-a367-464b48939cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:54.481663 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e5e19e-fa09-4aa6-a367-464b48939cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:54.483621 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e5e19e-fa09-4aa6-a367-464b48939cbc admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:54.510783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1305/2627] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:54.518264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b5a68-9c2e-4d98-828e-dff190331d6f admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:54.549190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b5a68-9c2e-4d98-828e-dff190331d6f admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:54.554635 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23817443-1663-453b-a2f4-915c7740ccc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:54.555423 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23817443-1663-453b-a2f4-915c7740ccc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:54.555589 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23817443-1663-453b-a2f4-915c7740ccc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:54.555872 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23817443-1663-453b-a2f4-915c7740ccc2 admin admin] PATH_INFO: `/v3/projects/8d86a0f556af4d7f97f2916df9bab632/users/61b570b0433149a3835457f95f4f3bb9/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:54.571261 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e5e19e-fa09-4aa6-a367-464b48939cbc admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:54.575980 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23817443-1663-453b-a2f4-915c7740ccc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d86a0f556af4d7f97f2916df9bab632, user_id=61b570b0433149a3835457f95f4f3bb9, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:54.579801 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23817443-1663-453b-a2f4-915c7740ccc2 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:54.583300 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e5e19e-fa09-4aa6-a367-464b48939cbc admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:54.590404 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475deb66-7ac0-4dda-bdfb-ef63a6b4cab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:54.591967 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475deb66-7ac0-4dda-bdfb-ef63a6b4cab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:54.592245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475deb66-7ac0-4dda-bdfb-ef63a6b4cab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:54.592698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475deb66-7ac0-4dda-bdfb-ef63a6b4cab8 admin admin] PATH_INFO: `/v3/users/5b249ec5ff4a4dbf8362d5974e97e19e` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:54.593956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475deb66-7ac0-4dda-bdfb-ef63a6b4cab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b249ec5ff4a4dbf8362d5974e97e19e)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:54.596109 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475deb66-7ac0-4dda-bdfb-ef63a6b4cab8 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:54.597264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1323/2628] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:54 2022] PUT /identity/v3/projects/8d86a0f556af4d7f97f2916df9bab632/users/61b570b0433149a3835457f95f4f3bb9/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:54.601636 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23817443-1663-453b-a2f4-915c7740ccc2 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:54.676830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-475deb66-7ac0-4dda-bdfb-ef63a6b4cab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b249ec5ff4a4dbf8362d5974e97e19e'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:54.677542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-475deb66-7ac0-4dda-bdfb-ef63a6b4cab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b249ec5ff4a4dbf8362d5974e97e19e'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:54.689477 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-475deb66-7ac0-4dda-bdfb-ef63a6b4cab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b249ec5ff4a4dbf8362d5974e97e19e'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:54.691001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-475deb66-7ac0-4dda-bdfb-ef63a6b4cab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b249ec5ff4a4dbf8362d5974e97e19e'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:54.712816 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-475deb66-7ac0-4dda-bdfb-ef63a6b4cab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b249ec5ff4a4dbf8362d5974e97e19e'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:54.715547 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:54.736542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1306/2629] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:54 2022] DELETE /identity/v3/users/5b249ec5ff4a4dbf8362d5974e97e19e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:54.738215 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f1cb4-f02a-4169-b6d9-d7c3986a2bb6 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:54.738510 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f1cb4-f02a-4169-b6d9-d7c3986a2bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:54.738789 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f1cb4-f02a-4169-b6d9-d7c3986a2bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:54.746498 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23817443-1663-453b-a2f4-915c7740ccc2 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:54.757432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:54.767793 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-433f1cb4-f02a-4169-b6d9-d7c3986a2bb6 None None] MFA Rules not processed for user `b687d38492594eab876740bdfb7db9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:54.769542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-433f1cb4-f02a-4169-b6d9-d7c3986a2bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:54.837252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f342d33-962f-48e7-bdb2-0ee9304582ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:54.837799 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f342d33-962f-48e7-bdb2-0ee9304582ef admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:54.838097 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f342d33-962f-48e7-bdb2-0ee9304582ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:54.838565 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f342d33-962f-48e7-bdb2-0ee9304582ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:54.839277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f342d33-962f-48e7-bdb2-0ee9304582ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:54.840981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f342d33-962f-48e7-bdb2-0ee9304582ef admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:54.853278 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1307/2630] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:54.853664 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1324/2631] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:19:54.854959 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bbc75-6dde-415e-9609-77eae233b114 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:54.855301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bbc75-6dde-415e-9609-77eae233b114 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:54.855726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bbc75-6dde-415e-9609-77eae233b114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:54.858955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f342d33-962f-48e7-bdb2-0ee9304582ef admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:54.861917 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5f342d33-962f-48e7-bdb2-0ee9304582ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:54.879190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f342d33-962f-48e7-bdb2-0ee9304582ef admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:54.883445 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-517bbc75-6dde-415e-9609-77eae233b114 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:54.884266 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d124d4-0583-472d-8ba8-9f3abbbf9e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:54.884956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-517bbc75-6dde-415e-9609-77eae233b114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:54.885147 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d124d4-0583-472d-8ba8-9f3abbbf9e10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:54.885433 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d124d4-0583-472d-8ba8-9f3abbbf9e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:54.885705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d124d4-0583-472d-8ba8-9f3abbbf9e10 admin admin] PATH_INFO: `/v3/projects/8d86a0f556af4d7f97f2916df9bab632/users/61b570b0433149a3835457f95f4f3bb9/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:54.887773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d124d4-0583-472d-8ba8-9f3abbbf9e10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d86a0f556af4d7f97f2916df9bab632, user_id=61b570b0433149a3835457f95f4f3bb9, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:54.890515 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d124d4-0583-472d-8ba8-9f3abbbf9e10 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:54.893999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1308/2632] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:54.905262 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1325/2633] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:54 2022] PUT /identity/v3/projects/8d86a0f556af4d7f97f2916df9bab632/users/61b570b0433149a3835457f95f4f3bb9/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:54.905916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bbc75-6dde-415e-9609-77eae233b114 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:54.907438 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-517bbc75-6dde-415e-9609-77eae233b114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:54.915429 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27801ba-2b86-42fe-94fe-4a4edd9bfacc None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:54.915725 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27801ba-2b86-42fe-94fe-4a4edd9bfacc None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:54.916002 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27801ba-2b86-42fe-94fe-4a4edd9bfacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:54.952264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-f27801ba-2b86-42fe-94fe-4a4edd9bfacc None None] MFA Rules not processed for user `61b570b0433149a3835457f95f4f3bb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:54.953635 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f27801ba-2b86-42fe-94fe-4a4edd9bfacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:54.978956 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bbc75-6dde-415e-9609-77eae233b114 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:55.032228 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1326/2634] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:19:55.043311 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27801ba-2b86-42fe-94fe-4a4edd9bfacc None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:55.043700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b1f48-15e2-4fc9-974e-95887f32aad9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:55.044929 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b1f48-15e2-4fc9-974e-95887f32aad9 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:55.045208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b1f48-15e2-4fc9-974e-95887f32aad9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:55.045477 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b1f48-15e2-4fc9-974e-95887f32aad9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:55.046119 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f27801ba-2b86-42fe-94fe-4a4edd9bfacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:55.110407 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27801ba-2b86-42fe-94fe-4a4edd9bfacc None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:55.114536 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=100 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:55.114762 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f80c4d-204f-4bdf-8dfc-4ff7b2464316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:55.114898 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f80c4d-204f-4bdf-8dfc-4ff7b2464316 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:55.115243 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f80c4d-204f-4bdf-8dfc-4ff7b2464316 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:55.115410 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f80c4d-204f-4bdf-8dfc-4ff7b2464316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:55.118913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b1f48-15e2-4fc9-974e-95887f32aad9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:55.120728 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b1f48-15e2-4fc9-974e-95887f32aad9 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:55.127349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f80c4d-204f-4bdf-8dfc-4ff7b2464316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:55.129032 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f80c4d-204f-4bdf-8dfc-4ff7b2464316 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:55.134278 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1309/2635] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:19:55.141872 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1327/2636] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:19:55.306777 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b1f48-15e2-4fc9-974e-95887f32aad9 service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:55.330786 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b1f48-15e2-4fc9-974e-95887f32aad9 service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:55.340789 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a8b71-f3b1-4006-9d4f-b5e787a43cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:55.343284 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a8b71-f3b1-4006-9d4f-b5e787a43cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:55.343697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a8b71-f3b1-4006-9d4f-b5e787a43cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:55.344117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a8b71-f3b1-4006-9d4f-b5e787a43cb0 admin admin] PATH_INFO: `/v3/users/afc747b842bf4046a5bda7f6a1ef9caf` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:55.346413 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a8b71-f3b1-4006-9d4f-b5e787a43cb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afc747b842bf4046a5bda7f6a1ef9caf)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:55.350251 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a8b71-f3b1-4006-9d4f-b5e787a43cb0 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:55.432967 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-199a8b71-f3b1-4006-9d4f-b5e787a43cb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afc747b842bf4046a5bda7f6a1ef9caf'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:55.434383 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-199a8b71-f3b1-4006-9d4f-b5e787a43cb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afc747b842bf4046a5bda7f6a1ef9caf'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:55.457432 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-199a8b71-f3b1-4006-9d4f-b5e787a43cb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afc747b842bf4046a5bda7f6a1ef9caf'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:55.458159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-199a8b71-f3b1-4006-9d4f-b5e787a43cb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afc747b842bf4046a5bda7f6a1ef9caf'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:55.474036 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-199a8b71-f3b1-4006-9d4f-b5e787a43cb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afc747b842bf4046a5bda7f6a1ef9caf'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:55.489860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1310/2637] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:55 2022] DELETE /identity/v3/users/afc747b842bf4046a5bda7f6a1ef9caf => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:55.494919 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc1a6b-5544-48bc-a799-771c1e9d696d None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:55.495454 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc1a6b-5544-48bc-a799-771c1e9d696d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:55.495573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc1a6b-5544-48bc-a799-771c1e9d696d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:55.517996 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-effc1a6b-5544-48bc-a799-771c1e9d696d None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:55.519239 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-effc1a6b-5544-48bc-a799-771c1e9d696d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:55.533296 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:55.606166 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1328/2638] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:19:55.616109 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a8b71-f3b1-4006-9d4f-b5e787a43cb0 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:55.618807 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-199a8b71-f3b1-4006-9d4f-b5e787a43cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:55.647796 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc1a6b-5544-48bc-a799-771c1e9d696d None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:55.649831 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-effc1a6b-5544-48bc-a799-771c1e9d696d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:55.672837 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc1a6b-5544-48bc-a799-771c1e9d696d None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:55.680159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89068550-fcd8-403f-8d87-72f3055eb006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:55.681519 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89068550-fcd8-403f-8d87-72f3055eb006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:55.681964 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89068550-fcd8-403f-8d87-72f3055eb006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:55.682416 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89068550-fcd8-403f-8d87-72f3055eb006 admin admin] PATH_INFO: `/v3/projects/a73675eec55e4819b1a742858fd66d90` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:55.684685 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89068550-fcd8-403f-8d87-72f3055eb006 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a73675eec55e4819b1a742858fd66d90)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:55.688104 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89068550-fcd8-403f-8d87-72f3055eb006 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:55.721255 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a8b71-f3b1-4006-9d4f-b5e787a43cb0 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:55.725119 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684eb4a7-ad36-45de-b984-2246c70ffb46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:55.725948 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684eb4a7-ad36-45de-b984-2246c70ffb46 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:55.726302 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684eb4a7-ad36-45de-b984-2246c70ffb46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:55.726643 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684eb4a7-ad36-45de-b984-2246c70ffb46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:55.739428 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684eb4a7-ad36-45de-b984-2246c70ffb46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:55.741032 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684eb4a7-ad36-45de-b984-2246c70ffb46 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:55.756770 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1311/2639] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:55.774501 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-89068550-fcd8-403f-8d87-72f3055eb006 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a73675eec55e4819b1a742858fd66d90', 'request_id': 'req-89068550-fcd8-403f-8d87-72f3055eb006'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:55.785216 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-89068550-fcd8-403f-8d87-72f3055eb006 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a73675eec55e4819b1a742858fd66d90', 'request_id': 'req-89068550-fcd8-403f-8d87-72f3055eb006'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:55.800703 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-89068550-fcd8-403f-8d87-72f3055eb006 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a73675eec55e4819b1a742858fd66d90', 'request_id': 'req-89068550-fcd8-403f-8d87-72f3055eb006'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:55.817330 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-89068550-fcd8-403f-8d87-72f3055eb006 admin admin] The token cache is being invalidate because project a73675eec55e4819b1a742858fd66d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:55.817750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-89068550-fcd8-403f-8d87-72f3055eb006 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:55.818700 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-89068550-fcd8-403f-8d87-72f3055eb006 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:55.823549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1329/2640] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:55 2022] DELETE /identity/v3/projects/a73675eec55e4819b1a742858fd66d90 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:55.825693 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684eb4a7-ad36-45de-b984-2246c70ffb46 service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:55.838632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684eb4a7-ad36-45de-b984-2246c70ffb46 service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:55.931451 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfafab-dadb-4ee4-93d9-dc6539462d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:55.932338 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfafab-dadb-4ee4-93d9-dc6539462d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:55.932779 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfafab-dadb-4ee4-93d9-dc6539462d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:55.933137 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfafab-dadb-4ee4-93d9-dc6539462d13 admin admin] PATH_INFO: `/v3/users/0d40fb09bdd24837b412b0c7ea59c93c` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:55.934640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dfafab-dadb-4ee4-93d9-dc6539462d13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d40fb09bdd24837b412b0c7ea59c93c)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:55.936836 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dfafab-dadb-4ee4-93d9-dc6539462d13 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:55.953462 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:56.057885 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a4dfafab-dadb-4ee4-93d9-dc6539462d13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d40fb09bdd24837b412b0c7ea59c93c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.060861 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a4dfafab-dadb-4ee4-93d9-dc6539462d13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d40fb09bdd24837b412b0c7ea59c93c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.087493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a4dfafab-dadb-4ee4-93d9-dc6539462d13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d40fb09bdd24837b412b0c7ea59c93c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.087493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a4dfafab-dadb-4ee4-93d9-dc6539462d13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d40fb09bdd24837b412b0c7ea59c93c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.115292 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba39821-207b-4691-bd79-87cf506e40e8 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:56.115685 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba39821-207b-4691-bd79-87cf506e40e8 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:56.115864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba39821-207b-4691-bd79-87cf506e40e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:56.154653 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-a4dfafab-dadb-4ee4-93d9-dc6539462d13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d40fb09bdd24837b412b0c7ea59c93c'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.160497 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-4ba39821-207b-4691-bd79-87cf506e40e8 None None] MFA Rules not processed for user `4370f8f474c2446e989d0d8c5b75e87a`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:56.161989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4ba39821-207b-4691-bd79-87cf506e40e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:56.189377 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1312/2641] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:55 2022] DELETE /identity/v3/users/0d40fb09bdd24837b412b0c7ea59c93c => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:56.281025 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1330/2642] 10.222.0.93 () {58 vars in 967 bytes} [Mon Aug 1 15:19:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:56.302081 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfafab-dadb-4ee4-93d9-dc6539462d13 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:56.304667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a4dfafab-dadb-4ee4-93d9-dc6539462d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:56.321284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba39821-207b-4691-bd79-87cf506e40e8 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:56.324864 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4ba39821-207b-4691-bd79-87cf506e40e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:56.346426 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba39821-207b-4691-bd79-87cf506e40e8 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:56.443409 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:56.467408 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfafab-dadb-4ee4-93d9-dc6539462d13 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:56.469334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86685e6-25f1-4ac0-ae8a-b23904d11086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:56.470015 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86685e6-25f1-4ac0-ae8a-b23904d11086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:56.470376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86685e6-25f1-4ac0-ae8a-b23904d11086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:56.470922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86685e6-25f1-4ac0-ae8a-b23904d11086 admin admin] PATH_INFO: `/v3/projects/4e57b8dc36d1451b96a1bf657bbaa8c8` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:56.472721 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86685e6-25f1-4ac0-ae8a-b23904d11086 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e57b8dc36d1451b96a1bf657bbaa8c8)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:56.475072 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55c631-6a51-447d-82aa-5cc76ca78ad8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4df37b02152c48509243b73ebbf49d8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:56.475961 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55c631-6a51-447d-82aa-5cc76ca78ad8 service swift] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:56.476406 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55c631-6a51-447d-82aa-5cc76ca78ad8 service swift] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:56.476881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55c631-6a51-447d-82aa-5cc76ca78ad8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:56.477389 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86685e6-25f1-4ac0-ae8a-b23904d11086 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:56.568803 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a86685e6-25f1-4ac0-ae8a-b23904d11086 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e57b8dc36d1451b96a1bf657bbaa8c8', 'request_id': 'req-a86685e6-25f1-4ac0-ae8a-b23904d11086'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.581948 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a86685e6-25f1-4ac0-ae8a-b23904d11086 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e57b8dc36d1451b96a1bf657bbaa8c8', 'request_id': 'req-a86685e6-25f1-4ac0-ae8a-b23904d11086'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.590214 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55c631-6a51-447d-82aa-5cc76ca78ad8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:56.594696 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55c631-6a51-447d-82aa-5cc76ca78ad8 service swift] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:56.597996 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a86685e6-25f1-4ac0-ae8a-b23904d11086 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e57b8dc36d1451b96a1bf657bbaa8c8', 'request_id': 'req-a86685e6-25f1-4ac0-ae8a-b23904d11086'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.613126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1313/2643] 10.222.0.93 () {60 vars in 1384 bytes} [Mon Aug 1 15:19:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:19:56.614313 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55c631-6a51-447d-82aa-5cc76ca78ad8 service swift] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:56.616870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-de55c631-6a51-447d-82aa-5cc76ca78ad8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:56.620084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a86685e6-25f1-4ac0-ae8a-b23904d11086 admin admin] The token cache is being invalidate because project 4e57b8dc36d1451b96a1bf657bbaa8c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:56.620387 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a86685e6-25f1-4ac0-ae8a-b23904d11086 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.621187 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a86685e6-25f1-4ac0-ae8a-b23904d11086 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.628793 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1331/2644] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:56 2022] DELETE /identity/v3/projects/4e57b8dc36d1451b96a1bf657bbaa8c8 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:56.629934 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86685e6-25f1-4ac0-ae8a-b23904d11086 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:56.632659 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a86685e6-25f1-4ac0-ae8a-b23904d11086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:56.643051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55c631-6a51-447d-82aa-5cc76ca78ad8 service swift] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:56.707601 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:56.755435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86685e6-25f1-4ac0-ae8a-b23904d11086 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:56.760301 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a929a-1237-482a-b2bb-06753e0a8727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:56.760999 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a929a-1237-482a-b2bb-06753e0a8727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:56.761281 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a929a-1237-482a-b2bb-06753e0a8727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:56.761583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a929a-1237-482a-b2bb-06753e0a8727 admin admin] PATH_INFO: `/v3/users/61b570b0433149a3835457f95f4f3bb9` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:56.762964 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a929a-1237-482a-b2bb-06753e0a8727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61b570b0433149a3835457f95f4f3bb9)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:56.765432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a929a-1237-482a-b2bb-06753e0a8727 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:56.788555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a85d45-7b81-4b4a-bc13-52823e0b8e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:56.790344 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a85d45-7b81-4b4a-bc13-52823e0b8e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:56.790937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a85d45-7b81-4b4a-bc13-52823e0b8e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:56.791276 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a85d45-7b81-4b4a-bc13-52823e0b8e3e admin admin] PATH_INFO: `/v3/projects/4002092a626241ee846ed7e46284a246` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:56.793829 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a85d45-7b81-4b4a-bc13-52823e0b8e3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4002092a626241ee846ed7e46284a246)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:56.800216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a85d45-7b81-4b4a-bc13-52823e0b8e3e admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:56.876176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5f1a929a-1237-482a-b2bb-06753e0a8727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61b570b0433149a3835457f95f4f3bb9'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.888303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5f1a929a-1237-482a-b2bb-06753e0a8727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61b570b0433149a3835457f95f4f3bb9'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.891490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-40a85d45-7b81-4b4a-bc13-52823e0b8e3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4002092a626241ee846ed7e46284a246', 'request_id': 'req-40a85d45-7b81-4b4a-bc13-52823e0b8e3e'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.900237 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-40a85d45-7b81-4b4a-bc13-52823e0b8e3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4002092a626241ee846ed7e46284a246', 'request_id': 'req-40a85d45-7b81-4b4a-bc13-52823e0b8e3e'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.902126 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5f1a929a-1237-482a-b2bb-06753e0a8727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61b570b0433149a3835457f95f4f3bb9'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.902975 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5f1a929a-1237-482a-b2bb-06753e0a8727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61b570b0433149a3835457f95f4f3bb9'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.904435 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5f1a929a-1237-482a-b2bb-06753e0a8727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61b570b0433149a3835457f95f4f3bb9'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.911746 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-40a85d45-7b81-4b4a-bc13-52823e0b8e3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4002092a626241ee846ed7e46284a246', 'request_id': 'req-40a85d45-7b81-4b4a-bc13-52823e0b8e3e'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.927246 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:56.933836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1332/2645] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:56 2022] DELETE /identity/v3/users/61b570b0433149a3835457f95f4f3bb9 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:56.941634 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-40a85d45-7b81-4b4a-bc13-52823e0b8e3e admin admin] The token cache is being invalidate because project 4002092a626241ee846ed7e46284a246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:56.942045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-40a85d45-7b81-4b4a-bc13-52823e0b8e3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.942865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-40a85d45-7b81-4b4a-bc13-52823e0b8e3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:56.945773 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1314/2646] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:56 2022] DELETE /identity/v3/projects/4002092a626241ee846ed7e46284a246 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:57.441363 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a929a-1237-482a-b2bb-06753e0a8727 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:57.443311 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5f1a929a-1237-482a-b2bb-06753e0a8727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:57.579194 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a929a-1237-482a-b2bb-06753e0a8727 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:57.587952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef2a40-4567-416c-97a0-8c2b006e1a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:57.589475 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef2a40-4567-416c-97a0-8c2b006e1a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:57.589894 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef2a40-4567-416c-97a0-8c2b006e1a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:57.590263 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef2a40-4567-416c-97a0-8c2b006e1a8a admin admin] PATH_INFO: `/v3/projects/8d86a0f556af4d7f97f2916df9bab632` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:57.591842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef2a40-4567-416c-97a0-8c2b006e1a8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d86a0f556af4d7f97f2916df9bab632)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:57.596818 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef2a40-4567-416c-97a0-8c2b006e1a8a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:57.682343 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fbef2a40-4567-416c-97a0-8c2b006e1a8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d86a0f556af4d7f97f2916df9bab632', 'request_id': 'req-fbef2a40-4567-416c-97a0-8c2b006e1a8a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:57.697563 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fbef2a40-4567-416c-97a0-8c2b006e1a8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d86a0f556af4d7f97f2916df9bab632', 'request_id': 'req-fbef2a40-4567-416c-97a0-8c2b006e1a8a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:57.711332 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fbef2a40-4567-416c-97a0-8c2b006e1a8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d86a0f556af4d7f97f2916df9bab632', 'request_id': 'req-fbef2a40-4567-416c-97a0-8c2b006e1a8a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:57.734990 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fbef2a40-4567-416c-97a0-8c2b006e1a8a admin admin] The token cache is being invalidate because project 8d86a0f556af4d7f97f2916df9bab632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:57.735482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fbef2a40-4567-416c-97a0-8c2b006e1a8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:57.736758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-fbef2a40-4567-416c-97a0-8c2b006e1a8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:57.739624 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1333/2647] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:57 2022] DELETE /identity/v3/projects/8d86a0f556af4d7f97f2916df9bab632 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:57.749688 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f837f2-ea42-44cd-ad21-d7468d35e5df None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:57.750352 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f837f2-ea42-44cd-ad21-d7468d35e5df None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:57.750352 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f837f2-ea42-44cd-ad21-d7468d35e5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:57.783168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-e6f837f2-ea42-44cd-ad21-d7468d35e5df None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:57.784545 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e6f837f2-ea42-44cd-ad21-d7468d35e5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:57.881427 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1315/2648] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:57.898229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab06bda-2c47-497f-a4bf-75d2970554eb None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:57.898229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab06bda-2c47-497f-a4bf-75d2970554eb None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:57.898229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab06bda-2c47-497f-a4bf-75d2970554eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:57.927172 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-cab06bda-2c47-497f-a4bf-75d2970554eb None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:57.928432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-cab06bda-2c47-497f-a4bf-75d2970554eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:57.937195 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1334/2649] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:57.942128 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f837f2-ea42-44cd-ad21-d7468d35e5df None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:57.954094 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f837f2-ea42-44cd-ad21-d7468d35e5df None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:57.959531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971197b6-5c1b-46e8-ae94-12559e9a40a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:57.960635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971197b6-5c1b-46e8-ae94-12559e9a40a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:57.960815 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971197b6-5c1b-46e8-ae94-12559e9a40a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:57.961173 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971197b6-5c1b-46e8-ae94-12559e9a40a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:57.962535 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971197b6-5c1b-46e8-ae94-12559e9a40a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:57.962844 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971197b6-5c1b-46e8-ae94-12559e9a40a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:57.965420 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971197b6-5c1b-46e8-ae94-12559e9a40a5 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:57.985265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1316/2650] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:19:57.991859 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab06bda-2c47-497f-a4bf-75d2970554eb None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:58.008666 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab06bda-2c47-497f-a4bf-75d2970554eb None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:58.017535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd492b7-7e7d-4063-b018-ab5878ecb64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:58.018218 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd492b7-7e7d-4063-b018-ab5878ecb64a admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:58.018520 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd492b7-7e7d-4063-b018-ab5878ecb64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:58.018790 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd492b7-7e7d-4063-b018-ab5878ecb64a admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:58.019741 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd492b7-7e7d-4063-b018-ab5878ecb64a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:58.022682 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd492b7-7e7d-4063-b018-ab5878ecb64a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:58.072205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1335/2651] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:57 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:19:58.079541 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971197b6-5c1b-46e8-ae94-12559e9a40a5 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:58.094282 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971197b6-5c1b-46e8-ae94-12559e9a40a5 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:58.160359 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:58.177435 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d1272-536b-47a6-84d1-95f4e374f558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:58.178811 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d1272-536b-47a6-84d1-95f4e374f558 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:58.179205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d1272-536b-47a6-84d1-95f4e374f558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:58.179583 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d1272-536b-47a6-84d1-95f4e374f558 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:58.180643 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d1272-536b-47a6-84d1-95f4e374f558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:58.182520 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d1272-536b-47a6-84d1-95f4e374f558 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:58.221016 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1317/2652] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:58 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:58.225715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd492b7-7e7d-4063-b018-ab5878ecb64a admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:58.238179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd492b7-7e7d-4063-b018-ab5878ecb64a admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:58.244834 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6778c-2387-4822-8997-f5469991219e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:58.245473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6778c-2387-4822-8997-f5469991219e admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:58.245876 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6778c-2387-4822-8997-f5469991219e admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:58.246356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6778c-2387-4822-8997-f5469991219e admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:58.247736 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6778c-2387-4822-8997-f5469991219e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:58.249624 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6778c-2387-4822-8997-f5469991219e admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:58.268126 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1336/2653] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:19:58.269570 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d1272-536b-47a6-84d1-95f4e374f558 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:58.284791 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d1272-536b-47a6-84d1-95f4e374f558 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:58.291528 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7175fe-6297-4d5a-8e45-94cb1570f990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:58.292299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7175fe-6297-4d5a-8e45-94cb1570f990 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:58.292895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7175fe-6297-4d5a-8e45-94cb1570f990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:58.293288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7175fe-6297-4d5a-8e45-94cb1570f990 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:58.294346 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7175fe-6297-4d5a-8e45-94cb1570f990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:58.296236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7175fe-6297-4d5a-8e45-94cb1570f990 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:58.321546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1318/2654] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:58.323112 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6778c-2387-4822-8997-f5469991219e admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:58.339871 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6778c-2387-4822-8997-f5469991219e admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:58.347308 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4ed1c-4370-408b-a989-01a4fccfc238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:58.348010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4ed1c-4370-408b-a989-01a4fccfc238 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:58.348289 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4ed1c-4370-408b-a989-01a4fccfc238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:58.349246 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4ed1c-4370-408b-a989-01a4fccfc238 admin admin] PATH_INFO: `/v3/projects/9393d7f4c2fe4f79b911b2f01c014a29/users/6121347799e347c7a6b9810e0eee733c/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:58.369281 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4ed1c-4370-408b-a989-01a4fccfc238 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9393d7f4c2fe4f79b911b2f01c014a29, user_id=6121347799e347c7a6b9810e0eee733c, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:58.373561 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4ed1c-4370-408b-a989-01a4fccfc238 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:58.400516 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1337/2655] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:58 2022] PUT /identity/v3/projects/9393d7f4c2fe4f79b911b2f01c014a29/users/6121347799e347c7a6b9810e0eee733c/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:58.413124 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9b37b-a88f-4815-b8c2-a256fb7195bd None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:58.413632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9b37b-a88f-4815-b8c2-a256fb7195bd None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:58.414117 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9b37b-a88f-4815-b8c2-a256fb7195bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:58.470941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-43f9b37b-a88f-4815-b8c2-a256fb7195bd None None] MFA Rules not processed for user `6121347799e347c7a6b9810e0eee733c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:58.473369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-43f9b37b-a88f-4815-b8c2-a256fb7195bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:58.604925 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1319/2656] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:58.610307 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4ed1c-4370-408b-a989-01a4fccfc238 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:58.651252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:58.693125 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4ed1c-4370-408b-a989-01a4fccfc238 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:58.699504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d3d0b-99bb-43e5-bf5f-0eecc76614c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:58.700284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d3d0b-99bb-43e5-bf5f-0eecc76614c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:58.700689 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d3d0b-99bb-43e5-bf5f-0eecc76614c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:58.701020 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d3d0b-99bb-43e5-bf5f-0eecc76614c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:58.701771 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948d3d0b-99bb-43e5-bf5f-0eecc76614c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:58.703851 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948d3d0b-99bb-43e5-bf5f-0eecc76614c8 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:58.739224 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1338/2657] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:58 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:58.743976 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9b37b-a88f-4815-b8c2-a256fb7195bd None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:58.759387 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9b37b-a88f-4815-b8c2-a256fb7195bd None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:58.854433 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def8f69-3dbc-482f-8cc6-deac4e7a5e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:58.855260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def8f69-3dbc-482f-8cc6-deac4e7a5e2a admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:58.855611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def8f69-3dbc-482f-8cc6-deac4e7a5e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:58.855955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def8f69-3dbc-482f-8cc6-deac4e7a5e2a admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:58.857362 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def8f69-3dbc-482f-8cc6-deac4e7a5e2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:58.860382 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def8f69-3dbc-482f-8cc6-deac4e7a5e2a admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:58.892692 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1320/2658] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:58 2022] POST /identity/v3/users => generated 294 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:58.895198 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d3d0b-99bb-43e5-bf5f-0eecc76614c8 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:58.912489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d3d0b-99bb-43e5-bf5f-0eecc76614c8 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:58.920038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c05713-aa86-468e-99d7-2b079c84f05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:58.920707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c05713-aa86-468e-99d7-2b079c84f05f admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:58.920981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c05713-aa86-468e-99d7-2b079c84f05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:58.921250 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c05713-aa86-468e-99d7-2b079c84f05f admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:58.921962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c05713-aa86-468e-99d7-2b079c84f05f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:58.923633 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c05713-aa86-468e-99d7-2b079c84f05f admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:58.938691 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1339/2659] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:58.943234 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def8f69-3dbc-482f-8cc6-deac4e7a5e2a admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:58.968392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def8f69-3dbc-482f-8cc6-deac4e7a5e2a admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:58.981472 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66052b4-901d-44e0-84b4-9bbdd2acdc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:58.982840 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66052b4-901d-44e0-84b4-9bbdd2acdc94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:58.983485 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66052b4-901d-44e0-84b4-9bbdd2acdc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:58.984130 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66052b4-901d-44e0-84b4-9bbdd2acdc94 admin admin] PATH_INFO: `/v3/projects/c2e34f1e070a4174b3cad2ca1e053780/users/7d7dc487d65f4f5ba43869edd4bbee54/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:59.015741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66052b4-901d-44e0-84b4-9bbdd2acdc94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e34f1e070a4174b3cad2ca1e053780, user_id=7d7dc487d65f4f5ba43869edd4bbee54, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:59.021825 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66052b4-901d-44e0-84b4-9bbdd2acdc94 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:59.042171 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1321/2660] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:58 2022] PUT /identity/v3/projects/c2e34f1e070a4174b3cad2ca1e053780/users/7d7dc487d65f4f5ba43869edd4bbee54/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:59.046839 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c05713-aa86-468e-99d7-2b079c84f05f admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:59.059954 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c05713-aa86-468e-99d7-2b079c84f05f admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:59.145987 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a11ad-f3a0-4e56-b34a-08454622cf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:59.146535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a11ad-f3a0-4e56-b34a-08454622cf33 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:59.146820 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a11ad-f3a0-4e56-b34a-08454622cf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:59.147116 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a11ad-f3a0-4e56-b34a-08454622cf33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:59.147820 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a11ad-f3a0-4e56-b34a-08454622cf33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:59.149692 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a11ad-f3a0-4e56-b34a-08454622cf33 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:59.163738 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1340/2661] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:59.168151 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66052b4-901d-44e0-84b4-9bbdd2acdc94 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:59.180448 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66052b4-901d-44e0-84b4-9bbdd2acdc94 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:59.186771 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3136cd-beb8-4e14-8076-167667bc9b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:59.187584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3136cd-beb8-4e14-8076-167667bc9b08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:59.187970 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3136cd-beb8-4e14-8076-167667bc9b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:59.188415 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3136cd-beb8-4e14-8076-167667bc9b08 admin admin] PATH_INFO: `/v3/projects/c2e34f1e070a4174b3cad2ca1e053780/users/7d7dc487d65f4f5ba43869edd4bbee54/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:59.190593 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3136cd-beb8-4e14-8076-167667bc9b08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e34f1e070a4174b3cad2ca1e053780, user_id=7d7dc487d65f4f5ba43869edd4bbee54, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:59.193386 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3136cd-beb8-4e14-8076-167667bc9b08 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:59.206150 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1322/2662] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:59 2022] PUT /identity/v3/projects/c2e34f1e070a4174b3cad2ca1e053780/users/7d7dc487d65f4f5ba43869edd4bbee54/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:59.217583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69a3a9-6542-47c2-8428-83cc091174e8 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:59.217921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69a3a9-6542-47c2-8428-83cc091174e8 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:59.218236 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69a3a9-6542-47c2-8428-83cc091174e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:59.253678 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-8e69a3a9-6542-47c2-8428-83cc091174e8 None None] MFA Rules not processed for user `7d7dc487d65f4f5ba43869edd4bbee54`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:59.255239 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-8e69a3a9-6542-47c2-8428-83cc091174e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:59.360811 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1341/2663] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:19:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:19:59.372677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3136cd-beb8-4e14-8076-167667bc9b08 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:59.374529 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ab3136cd-beb8-4e14-8076-167667bc9b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:59.386237 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:59.445397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3136cd-beb8-4e14-8076-167667bc9b08 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:59.449862 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bac94d-548f-436a-89a0-4044eeb8949a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:59.451082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bac94d-548f-436a-89a0-4044eeb8949a service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:59.451364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bac94d-548f-436a-89a0-4044eeb8949a service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:59.451698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bac94d-548f-436a-89a0-4044eeb8949a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:59.555018 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bac94d-548f-436a-89a0-4044eeb8949a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:59.558115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bac94d-548f-436a-89a0-4044eeb8949a service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:59.576652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1323/2664] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:20:02.354984 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69a3a9-6542-47c2-8428-83cc091174e8 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:02.378899 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:02.438098 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69a3a9-6542-47c2-8428-83cc091174e8 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:02.442867 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907ca16-15fb-4184-b71b-0e0bf590868f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:02.443533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907ca16-15fb-4184-b71b-0e0bf590868f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:02.443815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907ca16-15fb-4184-b71b-0e0bf590868f admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:02.444090 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907ca16-15fb-4184-b71b-0e0bf590868f admin admin] PATH_INFO: `/v3/users/6121347799e347c7a6b9810e0eee733c` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:02.445367 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5907ca16-15fb-4184-b71b-0e0bf590868f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6121347799e347c7a6b9810e0eee733c)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:02.447363 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5907ca16-15fb-4184-b71b-0e0bf590868f admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:02.517909 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5907ca16-15fb-4184-b71b-0e0bf590868f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6121347799e347c7a6b9810e0eee733c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:02.526519 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5907ca16-15fb-4184-b71b-0e0bf590868f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6121347799e347c7a6b9810e0eee733c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:02.536493 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5907ca16-15fb-4184-b71b-0e0bf590868f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6121347799e347c7a6b9810e0eee733c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:02.537252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5907ca16-15fb-4184-b71b-0e0bf590868f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6121347799e347c7a6b9810e0eee733c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:02.537874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5907ca16-15fb-4184-b71b-0e0bf590868f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6121347799e347c7a6b9810e0eee733c'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:02.562403 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1342/2665] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:20:02 2022] DELETE /identity/v3/users/6121347799e347c7a6b9810e0eee733c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:20:02.567454 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ebeb6-776a-44ed-9a0c-3b508634513f None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:02.568094 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ebeb6-776a-44ed-9a0c-3b508634513f None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:02.568094 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ebeb6-776a-44ed-9a0c-3b508634513f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:02.593178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-2f4ebeb6-776a-44ed-9a0c-3b508634513f None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:02.594413 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-2f4ebeb6-776a-44ed-9a0c-3b508634513f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:02.688110 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1324/2666] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:20:02.699734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907ca16-15fb-4184-b71b-0e0bf590868f admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:02.702259 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5907ca16-15fb-4184-b71b-0e0bf590868f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:02.783746 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907ca16-15fb-4184-b71b-0e0bf590868f admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:02.788601 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6135a-521c-411e-9694-bba40ed7b80d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:02.789374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6135a-521c-411e-9694-bba40ed7b80d service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:02.789743 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6135a-521c-411e-9694-bba40ed7b80d service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:02.790288 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6135a-521c-411e-9694-bba40ed7b80d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:02.801807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb6135a-521c-411e-9694-bba40ed7b80d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:02.803289 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb6135a-521c-411e-9694-bba40ed7b80d service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:02.814730 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1343/2667] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:20:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:20:03.169082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ebeb6-776a-44ed-9a0c-3b508634513f None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:03.171500 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-2f4ebeb6-776a-44ed-9a0c-3b508634513f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:03.191705 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ebeb6-776a-44ed-9a0c-3b508634513f None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:03.196572 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95032e79-847c-4826-a71c-a7da71083bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:03.197318 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95032e79-847c-4826-a71c-a7da71083bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:03.197595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95032e79-847c-4826-a71c-a7da71083bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:03.197873 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95032e79-847c-4826-a71c-a7da71083bfd admin admin] PATH_INFO: `/v3/projects/9393d7f4c2fe4f79b911b2f01c014a29` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:03.199214 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95032e79-847c-4826-a71c-a7da71083bfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9393d7f4c2fe4f79b911b2f01c014a29)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:03.201837 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95032e79-847c-4826-a71c-a7da71083bfd admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:03.274625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-95032e79-847c-4826-a71c-a7da71083bfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9393d7f4c2fe4f79b911b2f01c014a29', 'request_id': 'req-95032e79-847c-4826-a71c-a7da71083bfd'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:03.282160 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-95032e79-847c-4826-a71c-a7da71083bfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9393d7f4c2fe4f79b911b2f01c014a29', 'request_id': 'req-95032e79-847c-4826-a71c-a7da71083bfd'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:03.287571 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:03.290921 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-95032e79-847c-4826-a71c-a7da71083bfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9393d7f4c2fe4f79b911b2f01c014a29', 'request_id': 'req-95032e79-847c-4826-a71c-a7da71083bfd'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:03.309987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-95032e79-847c-4826-a71c-a7da71083bfd admin admin] The token cache is being invalidate because project 9393d7f4c2fe4f79b911b2f01c014a29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:20:03.310449 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-95032e79-847c-4826-a71c-a7da71083bfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:03.311168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-95032e79-847c-4826-a71c-a7da71083bfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:03.312787 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1325/2668] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:20:03 2022] DELETE /identity/v3/projects/9393d7f4c2fe4f79b911b2f01c014a29 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:20:03.319168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6135a-521c-411e-9694-bba40ed7b80d service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:03.339229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6135a-521c-411e-9694-bba40ed7b80d service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:03.403475 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:03.426204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bcc0cb-beb5-48ea-a587-e5ca8b49da3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:03.426768 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcc0cb-beb5-48ea-a587-e5ca8b49da3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:03.427048 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcc0cb-beb5-48ea-a587-e5ca8b49da3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:03.427320 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcc0cb-beb5-48ea-a587-e5ca8b49da3c admin admin] PATH_INFO: `/v3/users/7d7dc487d65f4f5ba43869edd4bbee54` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:03.428794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bcc0cb-beb5-48ea-a587-e5ca8b49da3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d7dc487d65f4f5ba43869edd4bbee54)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:03.430825 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bcc0cb-beb5-48ea-a587-e5ca8b49da3c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:03.490449 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-30bcc0cb-beb5-48ea-a587-e5ca8b49da3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d7dc487d65f4f5ba43869edd4bbee54'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:03.500038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-30bcc0cb-beb5-48ea-a587-e5ca8b49da3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d7dc487d65f4f5ba43869edd4bbee54'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:03.511356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-30bcc0cb-beb5-48ea-a587-e5ca8b49da3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d7dc487d65f4f5ba43869edd4bbee54'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:03.512023 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-30bcc0cb-beb5-48ea-a587-e5ca8b49da3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d7dc487d65f4f5ba43869edd4bbee54'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:03.512841 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-30bcc0cb-beb5-48ea-a587-e5ca8b49da3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d7dc487d65f4f5ba43869edd4bbee54'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:03.533125 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1344/2669] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:20:03 2022] DELETE /identity/v3/users/7d7dc487d65f4f5ba43869edd4bbee54 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:20:03.892034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95032e79-847c-4826-a71c-a7da71083bfd admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:03.893634 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-95032e79-847c-4826-a71c-a7da71083bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:03.929380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95032e79-847c-4826-a71c-a7da71083bfd admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:04.008540 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b4db7-393a-4b75-b666-5450d7b014c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:04.009406 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b4db7-393a-4b75-b666-5450d7b014c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:04.009821 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b4db7-393a-4b75-b666-5450d7b014c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:04.010259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b4db7-393a-4b75-b666-5450d7b014c1 admin admin] PATH_INFO: `/v3/projects/c2e34f1e070a4174b3cad2ca1e053780` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:04.011644 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b4db7-393a-4b75-b666-5450d7b014c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e34f1e070a4174b3cad2ca1e053780)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:04.014252 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b4db7-393a-4b75-b666-5450d7b014c1 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:04.076828 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0a4b4db7-393a-4b75-b666-5450d7b014c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e34f1e070a4174b3cad2ca1e053780', 'request_id': 'req-0a4b4db7-393a-4b75-b666-5450d7b014c1'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:04.084067 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0a4b4db7-393a-4b75-b666-5450d7b014c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e34f1e070a4174b3cad2ca1e053780', 'request_id': 'req-0a4b4db7-393a-4b75-b666-5450d7b014c1'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:04.090804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0a4b4db7-393a-4b75-b666-5450d7b014c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e34f1e070a4174b3cad2ca1e053780', 'request_id': 'req-0a4b4db7-393a-4b75-b666-5450d7b014c1'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:04.110789 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0a4b4db7-393a-4b75-b666-5450d7b014c1 admin admin] The token cache is being invalidate because project c2e34f1e070a4174b3cad2ca1e053780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:20:04.111375 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0a4b4db7-393a-4b75-b666-5450d7b014c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:04.112602 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0a4b4db7-393a-4b75-b666-5450d7b014c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:04.115165 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1326/2670] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:20:03 2022] DELETE /identity/v3/projects/c2e34f1e070a4174b3cad2ca1e053780 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:20:04.120826 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2c491-c6d5-4a63-8b0e-a74b4a49253e None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:04.121192 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2c491-c6d5-4a63-8b0e-a74b4a49253e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:04.121552 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2c491-c6d5-4a63-8b0e-a74b4a49253e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:04.144189 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-78e2c491-c6d5-4a63-8b0e-a74b4a49253e None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:04.145760 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-78e2c491-c6d5-4a63-8b0e-a74b4a49253e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:04.221995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1345/2671] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:20:04.236584 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c878551-c101-4570-9964-cdeb7cf04258 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:04.236871 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c878551-c101-4570-9964-cdeb7cf04258 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:04.237149 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c878551-c101-4570-9964-cdeb7cf04258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:04.261439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-2c878551-c101-4570-9964-cdeb7cf04258 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:04.262652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-2c878551-c101-4570-9964-cdeb7cf04258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:04.270701 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1327/2672] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:20:04.275179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2c491-c6d5-4a63-8b0e-a74b4a49253e None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:04.285418 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2c491-c6d5-4a63-8b0e-a74b4a49253e None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:04.290191 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48157d-1dfe-45fa-8242-e8be6e15430d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:04.290814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48157d-1dfe-45fa-8242-e8be6e15430d admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:04.291094 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48157d-1dfe-45fa-8242-e8be6e15430d admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:04.291369 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48157d-1dfe-45fa-8242-e8be6e15430d admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:04.292071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48157d-1dfe-45fa-8242-e8be6e15430d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:04.292380 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48157d-1dfe-45fa-8242-e8be6e15430d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:04.294543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48157d-1dfe-45fa-8242-e8be6e15430d admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:04.307445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1346/2673] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:04.312729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c878551-c101-4570-9964-cdeb7cf04258 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:04.322493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c878551-c101-4570-9964-cdeb7cf04258 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:04.327495 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa744df-843c-4aad-820f-307921fa9009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:04.328078 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa744df-843c-4aad-820f-307921fa9009 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:04.328357 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa744df-843c-4aad-820f-307921fa9009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:04.328665 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa744df-843c-4aad-820f-307921fa9009 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:04.329346 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa744df-843c-4aad-820f-307921fa9009 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:04.331274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa744df-843c-4aad-820f-307921fa9009 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:04.363885 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1328/2674] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:20:04 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:20:04.368550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48157d-1dfe-45fa-8242-e8be6e15430d admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:04.380233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48157d-1dfe-45fa-8242-e8be6e15430d admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:04.427798 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:04.447468 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2da609-8075-4791-bf69-6a14cad2acd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:04.448470 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2da609-8075-4791-bf69-6a14cad2acd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:04.449045 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2da609-8075-4791-bf69-6a14cad2acd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:04.449539 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2da609-8075-4791-bf69-6a14cad2acd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:04.450832 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2da609-8075-4791-bf69-6a14cad2acd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:04.454973 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2da609-8075-4791-bf69-6a14cad2acd3 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:04.496676 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1347/2675] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:20:04 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:20:04.497516 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa744df-843c-4aad-820f-307921fa9009 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:04.507427 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa744df-843c-4aad-820f-307921fa9009 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:04.512795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd41b16-44e5-4c64-90bb-fac8aa809dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:04.513490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd41b16-44e5-4c64-90bb-fac8aa809dfe admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:04.513847 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd41b16-44e5-4c64-90bb-fac8aa809dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:04.514632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd41b16-44e5-4c64-90bb-fac8aa809dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:04.514992 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd41b16-44e5-4c64-90bb-fac8aa809dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:04.516682 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd41b16-44e5-4c64-90bb-fac8aa809dfe admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:04.527868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1329/2676] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:20:04.532194 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2da609-8075-4791-bf69-6a14cad2acd3 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:04.548596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2da609-8075-4791-bf69-6a14cad2acd3 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:04.555177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2dc8fd-4837-4edf-9466-fb9b3c61c191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:04.556282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2dc8fd-4837-4edf-9466-fb9b3c61c191 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:04.556824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2dc8fd-4837-4edf-9466-fb9b3c61c191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:04.557343 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2dc8fd-4837-4edf-9466-fb9b3c61c191 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:04.558389 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2dc8fd-4837-4edf-9466-fb9b3c61c191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:04.561400 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2dc8fd-4837-4edf-9466-fb9b3c61c191 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:04.574799 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1348/2677] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:20:04.579408 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd41b16-44e5-4c64-90bb-fac8aa809dfe admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:04.590435 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd41b16-44e5-4c64-90bb-fac8aa809dfe admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:04.596164 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578067f8-1423-42d7-89e3-b370e39c106f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:04.596876 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578067f8-1423-42d7-89e3-b370e39c106f admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:04.597157 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578067f8-1423-42d7-89e3-b370e39c106f admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:04.597485 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578067f8-1423-42d7-89e3-b370e39c106f admin admin] PATH_INFO: `/v3/projects/9c1dffe73f924ee9aa7cffca0fa3d33f/users/aefedc120f4547e0964fb8386f9024dc/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:04.614941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578067f8-1423-42d7-89e3-b370e39c106f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c1dffe73f924ee9aa7cffca0fa3d33f, user_id=aefedc120f4547e0964fb8386f9024dc, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:04.617727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578067f8-1423-42d7-89e3-b370e39c106f admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:04.633971 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1330/2678] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:20:04 2022] PUT /identity/v3/projects/9c1dffe73f924ee9aa7cffca0fa3d33f/users/aefedc120f4547e0964fb8386f9024dc/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:20:04.639757 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871ab74-b104-4318-bacd-b12f79429c1f None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:04.640168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871ab74-b104-4318-bacd-b12f79429c1f None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:04.640530 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871ab74-b104-4318-bacd-b12f79429c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:04.663698 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-f871ab74-b104-4318-bacd-b12f79429c1f None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:04.665883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f871ab74-b104-4318-bacd-b12f79429c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:04.738937 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1349/2679] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:20:04.748998 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578067f8-1423-42d7-89e3-b370e39c106f admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:04.750933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-578067f8-1423-42d7-89e3-b370e39c106f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:04.766930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:04.830029 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578067f8-1423-42d7-89e3-b370e39c106f admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:04.834613 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536abf9-7bcb-4925-a77a-5dd1a9f1a453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:04.835429 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536abf9-7bcb-4925-a77a-5dd1a9f1a453 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:04.835745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536abf9-7bcb-4925-a77a-5dd1a9f1a453 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:04.836023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536abf9-7bcb-4925-a77a-5dd1a9f1a453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:04.847509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536abf9-7bcb-4925-a77a-5dd1a9f1a453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:04.849284 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536abf9-7bcb-4925-a77a-5dd1a9f1a453 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:04.861302 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1331/2680] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:20:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:20:05.574946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca304a60-bc92-432c-9cae-3ad3c4b8808f None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:05.575596 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca304a60-bc92-432c-9cae-3ad3c4b8808f None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:05.575693 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca304a60-bc92-432c-9cae-3ad3c4b8808f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:05.615901 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-ca304a60-bc92-432c-9cae-3ad3c4b8808f None None] MFA Rules not processed for user `aefedc120f4547e0964fb8386f9024dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:05.617356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ca304a60-bc92-432c-9cae-3ad3c4b8808f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:05.712891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1350/2681] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:20:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:20:05.728753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536abf9-7bcb-4925-a77a-5dd1a9f1a453 service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:05.731249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-4536abf9-7bcb-4925-a77a-5dd1a9f1a453 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:05.826194 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536abf9-7bcb-4925-a77a-5dd1a9f1a453 service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:05.830059 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc598df-8927-47c7-bc0a-c5d4d442f79b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:05.830713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc598df-8927-47c7-bc0a-c5d4d442f79b service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:05.830995 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc598df-8927-47c7-bc0a-c5d4d442f79b service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:05.831267 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc598df-8927-47c7-bc0a-c5d4d442f79b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:05.843088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc598df-8927-47c7-bc0a-c5d4d442f79b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:05.844977 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc598df-8927-47c7-bc0a-c5d4d442f79b service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:05.855952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1332/2682] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:20:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:20:07.342304 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5eea5-898e-4e02-85de-07f738a13493 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:07.343051 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5eea5-898e-4e02-85de-07f738a13493 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:07.343051 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5eea5-898e-4e02-85de-07f738a13493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:07.365590 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-efe5eea5-898e-4e02-85de-07f738a13493 None None] MFA Rules not processed for user `aefedc120f4547e0964fb8386f9024dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:07.366854 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-efe5eea5-898e-4e02-85de-07f738a13493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:07.374964 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1351/2683] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:20:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:20:15.850892 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=46 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:17.359171 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=74,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:23.941387 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc598df-8927-47c7-bc0a-c5d4d442f79b service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:23.943079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-fbc598df-8927-47c7-bc0a-c5d4d442f79b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:24.125903 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc598df-8927-47c7-bc0a-c5d4d442f79b service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:24.131390 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67656fc7-cbfa-4d09-aa79-3f12402422f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07e3e25e6a8466fb237dcb836aecda2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:24.132000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67656fc7-cbfa-4d09-aa79-3f12402422f8 service placement] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:24.132280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67656fc7-cbfa-4d09-aa79-3f12402422f8 service placement] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:24.132589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67656fc7-cbfa-4d09-aa79-3f12402422f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:24.134362 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-67656fc7-cbfa-4d09-aa79-3f12402422f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:24.146614 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67656fc7-cbfa-4d09-aa79-3f12402422f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:24.148214 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67656fc7-cbfa-4d09-aa79-3f12402422f8 service placement] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:24.160605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1333/2684] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:20:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:20:25.141042 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8359b-bd74-4866-b387-4536e367b41b None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:25.141519 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8359b-bd74-4866-b387-4536e367b41b None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:25.141736 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8359b-bd74-4866-b387-4536e367b41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:25.162027 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-05e8359b-bd74-4866-b387-4536e367b41b None None] MFA Rules not processed for user `aefedc120f4547e0964fb8386f9024dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:25.163248 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-05e8359b-bd74-4866-b387-4536e367b41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:25.170307 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1352/2685] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:20:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:20:34.155947 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=63 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:35.156763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:40.507494 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67656fc7-cbfa-4d09-aa79-3f12402422f8 service placement] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:40.531924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67656fc7-cbfa-4d09-aa79-3f12402422f8 service placement] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:40.535844 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0f01c-4686-4701-8520-af41572e23c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:40.536484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0f01c-4686-4701-8520-af41572e23c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:40.536764 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0f01c-4686-4701-8520-af41572e23c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:40.537035 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0f01c-4686-4701-8520-af41572e23c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:40.538842 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-3ee0f01c-4686-4701-8520-af41572e23c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:40.578948 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0f01c-4686-4701-8520-af41572e23c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:40.580759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0f01c-4686-4701-8520-af41572e23c0 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:40.606979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1334/2686] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:20:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:20:44.448149 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8359b-bd74-4866-b387-4536e367b41b None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:44.449957 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-05e8359b-bd74-4866-b387-4536e367b41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:44.465118 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8359b-bd74-4866-b387-4536e367b41b None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:44.472270 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f346c-7251-41f6-8444-37afc4fad064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:44.473003 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f346c-7251-41f6-8444-37afc4fad064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:44.473308 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f346c-7251-41f6-8444-37afc4fad064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:44.473576 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f346c-7251-41f6-8444-37afc4fad064 admin admin] PATH_INFO: `/v3/users/3c18437883ce42369c7563224c208523` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:44.474833 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f346c-7251-41f6-8444-37afc4fad064 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c18437883ce42369c7563224c208523)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:44.476824 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f346c-7251-41f6-8444-37afc4fad064 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:44.531644 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-382f346c-7251-41f6-8444-37afc4fad064 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c18437883ce42369c7563224c208523'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:44.539429 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-382f346c-7251-41f6-8444-37afc4fad064 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c18437883ce42369c7563224c208523'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:44.549736 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-382f346c-7251-41f6-8444-37afc4fad064 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c18437883ce42369c7563224c208523'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:44.550488 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-382f346c-7251-41f6-8444-37afc4fad064 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c18437883ce42369c7563224c208523'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:44.551262 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-382f346c-7251-41f6-8444-37afc4fad064 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c18437883ce42369c7563224c208523'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:44.574237 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1353/2687] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:20:44 2022] DELETE /identity/v3/users/3c18437883ce42369c7563224c208523 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:20:44.781860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0f01c-4686-4701-8520-af41572e23c0 service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:44.793313 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0f01c-4686-4701-8520-af41572e23c0 service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:44.866578 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f2a28-a841-4d84-9be9-36b28cfa76c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:44.867563 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f2a28-a841-4d84-9be9-36b28cfa76c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:44.868012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f2a28-a841-4d84-9be9-36b28cfa76c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:44.868405 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f2a28-a841-4d84-9be9-36b28cfa76c5 admin admin] PATH_INFO: `/v3/projects/8b18759847b44a698a092e4484b244d0` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:44.869988 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f2a28-a841-4d84-9be9-36b28cfa76c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b18759847b44a698a092e4484b244d0)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:44.871916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f2a28-a841-4d84-9be9-36b28cfa76c5 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:44.931978 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-bc3f2a28-a841-4d84-9be9-36b28cfa76c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b18759847b44a698a092e4484b244d0', 'request_id': 'req-bc3f2a28-a841-4d84-9be9-36b28cfa76c5'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:44.939450 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-bc3f2a28-a841-4d84-9be9-36b28cfa76c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b18759847b44a698a092e4484b244d0', 'request_id': 'req-bc3f2a28-a841-4d84-9be9-36b28cfa76c5'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:44.945640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-bc3f2a28-a841-4d84-9be9-36b28cfa76c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b18759847b44a698a092e4484b244d0', 'request_id': 'req-bc3f2a28-a841-4d84-9be9-36b28cfa76c5'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:44.959764 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-bc3f2a28-a841-4d84-9be9-36b28cfa76c5 admin admin] The token cache is being invalidate because project 8b18759847b44a698a092e4484b244d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:20:44.960046 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-bc3f2a28-a841-4d84-9be9-36b28cfa76c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:44.960725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-bc3f2a28-a841-4d84-9be9-36b28cfa76c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:44.962245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1335/2688] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:20:44 2022] DELETE /identity/v3/projects/8b18759847b44a698a092e4484b244d0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:20:44.968018 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961519b9-77c1-463d-92a3-31b1d2b0e540 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:44.968316 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961519b9-77c1-463d-92a3-31b1d2b0e540 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:44.968662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961519b9-77c1-463d-92a3-31b1d2b0e540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.061305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-961519b9-77c1-463d-92a3-31b1d2b0e540 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:45.063349 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-961519b9-77c1-463d-92a3-31b1d2b0e540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:45.132722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1354/2689] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:20:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:20:45.141473 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf0852-b7de-4036-9e53-c7c7259e0533 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.141771 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf0852-b7de-4036-9e53-c7c7259e0533 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.142027 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf0852-b7de-4036-9e53-c7c7259e0533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.164990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-fbbf0852-b7de-4036-9e53-c7c7259e0533 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:45.166342 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-fbbf0852-b7de-4036-9e53-c7c7259e0533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:45.174657 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1336/2690] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:20:45.180750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961519b9-77c1-463d-92a3-31b1d2b0e540 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:45.192335 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961519b9-77c1-463d-92a3-31b1d2b0e540 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:45.198709 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997aef65-3304-4f9c-9e9c-c35e520b562b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:45.204314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997aef65-3304-4f9c-9e9c-c35e520b562b admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.204314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997aef65-3304-4f9c-9e9c-c35e520b562b admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.204314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997aef65-3304-4f9c-9e9c-c35e520b562b admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.204314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997aef65-3304-4f9c-9e9c-c35e520b562b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:20:45.204314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997aef65-3304-4f9c-9e9c-c35e520b562b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:45.204314 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997aef65-3304-4f9c-9e9c-c35e520b562b admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:45.224143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1355/2691] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:20:45.229096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf0852-b7de-4036-9e53-c7c7259e0533 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:45.241543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf0852-b7de-4036-9e53-c7c7259e0533 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:45.248147 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e027cc-8ee6-4644-b247-c6351f81308d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:45.248864 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e027cc-8ee6-4644-b247-c6351f81308d admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.249153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e027cc-8ee6-4644-b247-c6351f81308d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.249473 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e027cc-8ee6-4644-b247-c6351f81308d admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.250163 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e027cc-8ee6-4644-b247-c6351f81308d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:45.252256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e027cc-8ee6-4644-b247-c6351f81308d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:45.293295 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1337/2692] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:20:45 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:20:45.298451 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997aef65-3304-4f9c-9e9c-c35e520b562b admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:45.310688 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997aef65-3304-4f9c-9e9c-c35e520b562b admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:45.342413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:45.380177 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce6366-d9af-49e6-a3fb-5e1aa238f243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:45.380842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce6366-d9af-49e6-a3fb-5e1aa238f243 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.381138 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce6366-d9af-49e6-a3fb-5e1aa238f243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.381393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce6366-d9af-49e6-a3fb-5e1aa238f243 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.382084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce6366-d9af-49e6-a3fb-5e1aa238f243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:45.384034 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce6366-d9af-49e6-a3fb-5e1aa238f243 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:45.409220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1356/2693] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:20:45 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:20:45.417172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e027cc-8ee6-4644-b247-c6351f81308d admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:45.433760 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e027cc-8ee6-4644-b247-c6351f81308d admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:45.440509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0e9e2-68f7-4280-a00a-d27bd7112677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:45.441265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0e9e2-68f7-4280-a00a-d27bd7112677 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.441608 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0e9e2-68f7-4280-a00a-d27bd7112677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.441882 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0e9e2-68f7-4280-a00a-d27bd7112677 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.442577 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0e9e2-68f7-4280-a00a-d27bd7112677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:45.444225 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0e9e2-68f7-4280-a00a-d27bd7112677 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:45.450341 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:45.458493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1338/2694] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:20:45.464564 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce6366-d9af-49e6-a3fb-5e1aa238f243 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:45.475625 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce6366-d9af-49e6-a3fb-5e1aa238f243 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:45.482126 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f8d01-806b-4a9b-9d08-73b3f1e27753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:45.482749 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f8d01-806b-4a9b-9d08-73b3f1e27753 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.483032 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f8d01-806b-4a9b-9d08-73b3f1e27753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.483308 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f8d01-806b-4a9b-9d08-73b3f1e27753 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.484012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f8d01-806b-4a9b-9d08-73b3f1e27753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:45.485684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f8d01-806b-4a9b-9d08-73b3f1e27753 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:45.500070 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1357/2695] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:20:45.505002 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0e9e2-68f7-4280-a00a-d27bd7112677 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:45.517126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0e9e2-68f7-4280-a00a-d27bd7112677 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:45.521899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0366c-f868-480e-a93d-d677c1fcf835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:45.522567 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0366c-f868-480e-a93d-d677c1fcf835 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.522851 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0366c-f868-480e-a93d-d677c1fcf835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.523759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0366c-f868-480e-a93d-d677c1fcf835 admin admin] PATH_INFO: `/v3/projects/f677d43502b044c981609fe0f0aed425/users/9673eb00ccc24fbaa495ddcbaf6d3076/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.541138 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0366c-f868-480e-a93d-d677c1fcf835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f677d43502b044c981609fe0f0aed425, user_id=9673eb00ccc24fbaa495ddcbaf6d3076, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:45.545994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0366c-f868-480e-a93d-d677c1fcf835 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:45.561855 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1339/2696] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:20:45 2022] PUT /identity/v3/projects/f677d43502b044c981609fe0f0aed425/users/9673eb00ccc24fbaa495ddcbaf6d3076/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:20:45.569200 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae4636-400c-4e29-ba09-4af130ebbb53 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.569484 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae4636-400c-4e29-ba09-4af130ebbb53 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.569739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae4636-400c-4e29-ba09-4af130ebbb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.592091 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-f0ae4636-400c-4e29-ba09-4af130ebbb53 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:45.593662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f0ae4636-400c-4e29-ba09-4af130ebbb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:45.677829 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1358/2697] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:20:45.689362 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0366c-f868-480e-a93d-d677c1fcf835 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:45.691218 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c6a0366c-f868-480e-a93d-d677c1fcf835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:45.768103 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0366c-f868-480e-a93d-d677c1fcf835 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:45.772177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e7af3-5898-4391-b4ad-1440ac4ae1b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:45.772996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e7af3-5898-4391-b4ad-1440ac4ae1b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.773335 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e7af3-5898-4391-b4ad-1440ac4ae1b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.773611 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e7af3-5898-4391-b4ad-1440ac4ae1b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.784771 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e7af3-5898-4391-b4ad-1440ac4ae1b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:45.787192 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e7af3-5898-4391-b4ad-1440ac4ae1b1 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:45.799475 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1340/2698] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:20:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:20:46.133244 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01128582-be86-4768-8a13-eccfd097475a None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.134089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01128582-be86-4768-8a13-eccfd097475a None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.135280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01128582-be86-4768-8a13-eccfd097475a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.182498 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-01128582-be86-4768-8a13-eccfd097475a None None] MFA Rules not processed for user `4370f8f474c2446e989d0d8c5b75e87a`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:46.183735 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-01128582-be86-4768-8a13-eccfd097475a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:46.288525 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1359/2699] 10.222.0.93 () {58 vars in 1009 bytes} [Mon Aug 1 15:20:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:20:46.319199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e7af3-5898-4391-b4ad-1440ac4ae1b1 service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:46.321619 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b04e7af3-5898-4391-b4ad-1440ac4ae1b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:46.429643 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e7af3-5898-4391-b4ad-1440ac4ae1b1 service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:46.435632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956eb4fa-948d-4228-9505-cc7435de4d91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4df37b02152c48509243b73ebbf49d8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:46.436313 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956eb4fa-948d-4228-9505-cc7435de4d91 service swift] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.436631 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956eb4fa-948d-4228-9505-cc7435de4d91 service swift] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.436891 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956eb4fa-948d-4228-9505-cc7435de4d91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.446983 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956eb4fa-948d-4228-9505-cc7435de4d91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:46.449069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956eb4fa-948d-4228-9505-cc7435de4d91 service swift] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:46.461382 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1341/2700] 10.222.0.93 () {60 vars in 1384 bytes} [Mon Aug 1 15:20:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:20:52.195077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f58320-decc-4615-bfe0-2c0a4d5e52b9 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:52.197620 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f58320-decc-4615-bfe0-2c0a4d5e52b9 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:52.198715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f58320-decc-4615-bfe0-2c0a4d5e52b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:52.251089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-10f58320-decc-4615-bfe0-2c0a4d5e52b9 None None] MFA Rules not processed for user `9673eb00ccc24fbaa495ddcbaf6d3076`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:52.253064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-10f58320-decc-4615-bfe0-2c0a4d5e52b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:52.300489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:52.383266 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1360/2701] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:20:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:20:52.411518 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956eb4fa-948d-4228-9505-cc7435de4d91 service swift] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:52.413777 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-956eb4fa-948d-4228-9505-cc7435de4d91 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:52.580669 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:52.647374 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956eb4fa-948d-4228-9505-cc7435de4d91 service swift] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:52.654676 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a39a6d-c9cf-4e94-9a81-ae7fbe45d5e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:52.655407 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a39a6d-c9cf-4e94-9a81-ae7fbe45d5e3 service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:52.655755 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a39a6d-c9cf-4e94-9a81-ae7fbe45d5e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:52.656089 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a39a6d-c9cf-4e94-9a81-ae7fbe45d5e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:52.667913 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a39a6d-c9cf-4e94-9a81-ae7fbe45d5e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:52.670197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a39a6d-c9cf-4e94-9a81-ae7fbe45d5e3 service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:52.684899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1342/2702] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:20:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:20:54.487715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dd7b8-08aa-450e-b613-8a6269e18e6e None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:54.488335 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dd7b8-08aa-450e-b613-8a6269e18e6e None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:54.488450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dd7b8-08aa-450e-b613-8a6269e18e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:54.528939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-e67dd7b8-08aa-450e-b613-8a6269e18e6e None None] MFA Rules not processed for user `9673eb00ccc24fbaa495ddcbaf6d3076`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:54.530201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e67dd7b8-08aa-450e-b613-8a6269e18e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:54.539083 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1361/2703] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:20:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:02.676695 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=17 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:04.522238 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=18 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:05.237931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750eca92-db30-471a-8bfb-37677471ebd2 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:05.238486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750eca92-db30-471a-8bfb-37677471ebd2 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:05.238486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750eca92-db30-471a-8bfb-37677471ebd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:05.260652 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-750eca92-db30-471a-8bfb-37677471ebd2 None None] MFA Rules not processed for user `4370f8f474c2446e989d0d8c5b75e87a`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:05.262333 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-750eca92-db30-471a-8bfb-37677471ebd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:05.269935 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1343/2704] 10.222.0.93 () {58 vars in 967 bytes} [Mon Aug 1 15:21:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:05.284320 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67dd7b8-08aa-450e-b613-8a6269e18e6e None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:05.299014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67dd7b8-08aa-450e-b613-8a6269e18e6e None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:05.304365 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d01655-aae9-4511-905e-5b12e4ded639 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4df37b02152c48509243b73ebbf49d8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:05.305038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d01655-aae9-4511-905e-5b12e4ded639 service swift] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:05.305320 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d01655-aae9-4511-905e-5b12e4ded639 service swift] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:05.305607 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d01655-aae9-4511-905e-5b12e4ded639 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:05.315286 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d01655-aae9-4511-905e-5b12e4ded639 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:05.320989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d01655-aae9-4511-905e-5b12e4ded639 service swift] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:05.325967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1362/2705] 10.222.0.93 () {60 vars in 1384 bytes} [Mon Aug 1 15:21:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:21:07.351502 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750eca92-db30-471a-8bfb-37677471ebd2 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:07.353932 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-750eca92-db30-471a-8bfb-37677471ebd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:07.373695 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750eca92-db30-471a-8bfb-37677471ebd2 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:07.378908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5d7a9-de64-4670-bd75-2a880f9e6c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:07.379555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5d7a9-de64-4670-bd75-2a880f9e6c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:07.379830 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5d7a9-de64-4670-bd75-2a880f9e6c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:07.380115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5d7a9-de64-4670-bd75-2a880f9e6c4e admin admin] PATH_INFO: `/v3/users/aefedc120f4547e0964fb8386f9024dc` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:07.381321 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5d7a9-de64-4670-bd75-2a880f9e6c4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=aefedc120f4547e0964fb8386f9024dc)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:07.383344 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5d7a9-de64-4670-bd75-2a880f9e6c4e admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:07.439161 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1fc5d7a9-de64-4670-bd75-2a880f9e6c4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aefedc120f4547e0964fb8386f9024dc'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:07.439993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1fc5d7a9-de64-4670-bd75-2a880f9e6c4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aefedc120f4547e0964fb8386f9024dc'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:07.448422 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1fc5d7a9-de64-4670-bd75-2a880f9e6c4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aefedc120f4547e0964fb8386f9024dc'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:07.449760 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1fc5d7a9-de64-4670-bd75-2a880f9e6c4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aefedc120f4547e0964fb8386f9024dc'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:07.462723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1fc5d7a9-de64-4670-bd75-2a880f9e6c4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aefedc120f4547e0964fb8386f9024dc'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:07.476036 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1344/2706] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:21:07 2022] DELETE /identity/v3/users/aefedc120f4547e0964fb8386f9024dc => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:21:07.675739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d01655-aae9-4511-905e-5b12e4ded639 service swift] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:07.688265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d01655-aae9-4511-905e-5b12e4ded639 service swift] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:07.778504 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932cb72-1e2d-40c0-b113-b2c796ca06a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:07.779241 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932cb72-1e2d-40c0-b113-b2c796ca06a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:07.779512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932cb72-1e2d-40c0-b113-b2c796ca06a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:07.779782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932cb72-1e2d-40c0-b113-b2c796ca06a6 admin admin] PATH_INFO: `/v3/projects/9c1dffe73f924ee9aa7cffca0fa3d33f` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:07.782697 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932cb72-1e2d-40c0-b113-b2c796ca06a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c1dffe73f924ee9aa7cffca0fa3d33f)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:07.784779 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932cb72-1e2d-40c0-b113-b2c796ca06a6 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:07.842593 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6932cb72-1e2d-40c0-b113-b2c796ca06a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c1dffe73f924ee9aa7cffca0fa3d33f', 'request_id': 'req-6932cb72-1e2d-40c0-b113-b2c796ca06a6'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:07.850770 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6932cb72-1e2d-40c0-b113-b2c796ca06a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c1dffe73f924ee9aa7cffca0fa3d33f', 'request_id': 'req-6932cb72-1e2d-40c0-b113-b2c796ca06a6'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:07.861684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6932cb72-1e2d-40c0-b113-b2c796ca06a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c1dffe73f924ee9aa7cffca0fa3d33f', 'request_id': 'req-6932cb72-1e2d-40c0-b113-b2c796ca06a6'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:07.880573 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6932cb72-1e2d-40c0-b113-b2c796ca06a6 admin admin] The token cache is being invalidate because project 9c1dffe73f924ee9aa7cffca0fa3d33f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:21:07.880967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6932cb72-1e2d-40c0-b113-b2c796ca06a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:07.881962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-6932cb72-1e2d-40c0-b113-b2c796ca06a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:07.887822 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1363/2707] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:21:07 2022] DELETE /identity/v3/projects/9c1dffe73f924ee9aa7cffca0fa3d33f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:21:07.889569 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005c9a5-3dae-4aee-8b03-cdca02f5786b None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:07.889798 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005c9a5-3dae-4aee-8b03-cdca02f5786b None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:07.890074 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005c9a5-3dae-4aee-8b03-cdca02f5786b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:07.913405 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-1005c9a5-3dae-4aee-8b03-cdca02f5786b None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:07.916406 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-1005c9a5-3dae-4aee-8b03-cdca02f5786b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:07.987310 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1345/2708] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:21:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:07.995263 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9a81c-0d84-44d6-882d-26391e2afd75 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:07.995620 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9a81c-0d84-44d6-882d-26391e2afd75 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:07.995966 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9a81c-0d84-44d6-882d-26391e2afd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:08.018133 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-fbc9a81c-0d84-44d6-882d-26391e2afd75 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:08.019574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-fbc9a81c-0d84-44d6-882d-26391e2afd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:08.029372 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1364/2709] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:21:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:08.033872 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005c9a5-3dae-4aee-8b03-cdca02f5786b None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:08.044090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005c9a5-3dae-4aee-8b03-cdca02f5786b None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:08.049035 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e326d3-3efb-48eb-a2da-7c1d2d6e3282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:08.049666 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e326d3-3efb-48eb-a2da-7c1d2d6e3282 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:08.049949 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e326d3-3efb-48eb-a2da-7c1d2d6e3282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:08.050250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e326d3-3efb-48eb-a2da-7c1d2d6e3282 admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:08.051088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e326d3-3efb-48eb-a2da-7c1d2d6e3282 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:08.051399 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e326d3-3efb-48eb-a2da-7c1d2d6e3282 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:08.053046 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e326d3-3efb-48eb-a2da-7c1d2d6e3282 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:08.066364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1346/2710] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:08.070714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9a81c-0d84-44d6-882d-26391e2afd75 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:08.081521 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9a81c-0d84-44d6-882d-26391e2afd75 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:08.087933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaaa693-d886-4828-be55-a1f4eee0a2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:08.088604 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaaa693-d886-4828-be55-a1f4eee0a2f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:08.088951 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaaa693-d886-4828-be55-a1f4eee0a2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:08.089205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaaa693-d886-4828-be55-a1f4eee0a2f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:08.090921 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaaa693-d886-4828-be55-a1f4eee0a2f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:08.092903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaaa693-d886-4828-be55-a1f4eee0a2f3 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:08.127922 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1365/2711] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:21:08 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:21:08.132177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e326d3-3efb-48eb-a2da-7c1d2d6e3282 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:08.143477 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e326d3-3efb-48eb-a2da-7c1d2d6e3282 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:08.210402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:08.224957 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efa0d1-f338-471a-86b2-1b813fa70772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:08.226279 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efa0d1-f338-471a-86b2-1b813fa70772 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:08.226931 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efa0d1-f338-471a-86b2-1b813fa70772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:08.227546 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efa0d1-f338-471a-86b2-1b813fa70772 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:08.228806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efa0d1-f338-471a-86b2-1b813fa70772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:08.232602 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efa0d1-f338-471a-86b2-1b813fa70772 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:08.258599 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1347/2712] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:21:08 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:21:08.264222 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaaa693-d886-4828-be55-a1f4eee0a2f3 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:08.286284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaaa693-d886-4828-be55-a1f4eee0a2f3 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:08.292540 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62277d-be1e-45aa-8b63-b7385613319d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:08.293450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62277d-be1e-45aa-8b63-b7385613319d admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:08.293672 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62277d-be1e-45aa-8b63-b7385613319d admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:08.293955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62277d-be1e-45aa-8b63-b7385613319d admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:08.294757 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f62277d-be1e-45aa-8b63-b7385613319d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:08.296583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f62277d-be1e-45aa-8b63-b7385613319d admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:08.308382 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1366/2713] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:21:08.313203 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efa0d1-f338-471a-86b2-1b813fa70772 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:08.328734 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efa0d1-f338-471a-86b2-1b813fa70772 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:08.334015 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ef27c-63f8-49ef-ab79-d8f5b7811cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:08.334721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ef27c-63f8-49ef-ab79-d8f5b7811cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:08.335179 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ef27c-63f8-49ef-ab79-d8f5b7811cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:08.335358 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ef27c-63f8-49ef-ab79-d8f5b7811cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:08.336019 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ef27c-63f8-49ef-ab79-d8f5b7811cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:08.337721 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ef27c-63f8-49ef-ab79-d8f5b7811cd5 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:08.351766 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1348/2714] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:21:08.356312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62277d-be1e-45aa-8b63-b7385613319d admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:08.367035 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62277d-be1e-45aa-8b63-b7385613319d admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:08.373062 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f1138-2881-4d4d-872e-b86403de8ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:08.373773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f1138-2881-4d4d-872e-b86403de8ddf admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:08.374725 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f1138-2881-4d4d-872e-b86403de8ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:08.374725 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f1138-2881-4d4d-872e-b86403de8ddf admin admin] PATH_INFO: `/v3/projects/d9a60de4a5a042149f7819eb10552a43/users/858c64a842654b6680e49e6b60dcc9e8/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:08.394167 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f1138-2881-4d4d-872e-b86403de8ddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9a60de4a5a042149f7819eb10552a43, user_id=858c64a842654b6680e49e6b60dcc9e8, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:08.397290 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f1138-2881-4d4d-872e-b86403de8ddf admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:08.413244 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1367/2715] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:21:08 2022] PUT /identity/v3/projects/d9a60de4a5a042149f7819eb10552a43/users/858c64a842654b6680e49e6b60dcc9e8/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:21:08.482952 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b641b3f-a7f4-487f-b059-4ac4403c61c6 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:08.483172 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b641b3f-a7f4-487f-b059-4ac4403c61c6 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:08.483406 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b641b3f-a7f4-487f-b059-4ac4403c61c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:08.523841 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-3b641b3f-a7f4-487f-b059-4ac4403c61c6 None None] MFA Rules not processed for user `858c64a842654b6680e49e6b60dcc9e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:08.525121 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-3b641b3f-a7f4-487f-b059-4ac4403c61c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:08.596801 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1349/2716] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:21:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:08.607535 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f1138-2881-4d4d-872e-b86403de8ddf admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:08.609295 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ec0f1138-2881-4d4d-872e-b86403de8ddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:08.629161 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:08.705739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f1138-2881-4d4d-872e-b86403de8ddf admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:08.710462 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d55b82-33cc-44f7-98a4-459209cc6734 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:08.711334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d55b82-33cc-44f7-98a4-459209cc6734 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:08.711613 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d55b82-33cc-44f7-98a4-459209cc6734 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:08.711916 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d55b82-33cc-44f7-98a4-459209cc6734 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:08.721485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d55b82-33cc-44f7-98a4-459209cc6734 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:08.722969 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d55b82-33cc-44f7-98a4-459209cc6734 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:08.734217 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1368/2717] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:21:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:21:18.593761 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=42,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:18.728261 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=29 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:18.750408 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b641b3f-a7f4-487f-b059-4ac4403c61c6 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:18.819444 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b641b3f-a7f4-487f-b059-4ac4403c61c6 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:18.824259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d4934-9925-4711-9686-0f90ce448759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:18.824921 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d4934-9925-4711-9686-0f90ce448759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:18.825205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d4934-9925-4711-9686-0f90ce448759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:18.825477 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d4934-9925-4711-9686-0f90ce448759 admin admin] PATH_INFO: `/v3/users/858c64a842654b6680e49e6b60dcc9e8` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:18.826633 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d4934-9925-4711-9686-0f90ce448759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=858c64a842654b6680e49e6b60dcc9e8)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:18.828635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d4934-9925-4711-9686-0f90ce448759 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:18.880290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1c2d4934-9925-4711-9686-0f90ce448759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '858c64a842654b6680e49e6b60dcc9e8'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:18.880970 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1c2d4934-9925-4711-9686-0f90ce448759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '858c64a842654b6680e49e6b60dcc9e8'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:18.888726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1c2d4934-9925-4711-9686-0f90ce448759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '858c64a842654b6680e49e6b60dcc9e8'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:18.889479 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1c2d4934-9925-4711-9686-0f90ce448759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '858c64a842654b6680e49e6b60dcc9e8'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:18.901661 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-1c2d4934-9925-4711-9686-0f90ce448759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '858c64a842654b6680e49e6b60dcc9e8'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:18.915303 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1350/2718] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:21:18 2022] DELETE /identity/v3/users/858c64a842654b6680e49e6b60dcc9e8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:21:18.920528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab34ee-139f-470f-b0f4-ac3b13cfe343 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:18.920817 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab34ee-139f-470f-b0f4-ac3b13cfe343 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:18.921096 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab34ee-139f-470f-b0f4-ac3b13cfe343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:18.943913 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-91ab34ee-139f-470f-b0f4-ac3b13cfe343 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:18.945181 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-91ab34ee-139f-470f-b0f4-ac3b13cfe343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:19.006210 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1369/2719] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:19.016496 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d4934-9925-4711-9686-0f90ce448759 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:19.017902 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-1c2d4934-9925-4711-9686-0f90ce448759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:19.078108 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d4934-9925-4711-9686-0f90ce448759 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:19.082998 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d5bd6-e4c6-4db7-a391-112aa6930dee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:19.084011 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d5bd6-e4c6-4db7-a391-112aa6930dee service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:19.084291 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d5bd6-e4c6-4db7-a391-112aa6930dee service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:19.084594 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d5bd6-e4c6-4db7-a391-112aa6930dee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:19.095383 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d5bd6-e4c6-4db7-a391-112aa6930dee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:19.096863 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d5bd6-e4c6-4db7-a391-112aa6930dee service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:19.111154 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1351/2720] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:21:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:21:19.610303 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab34ee-139f-470f-b0f4-ac3b13cfe343 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:19.612100 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-91ab34ee-139f-470f-b0f4-ac3b13cfe343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:19.634983 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab34ee-139f-470f-b0f4-ac3b13cfe343 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:19.639986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0e42e-9e98-4a53-9944-e7665e55c323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:19.640682 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0e42e-9e98-4a53-9944-e7665e55c323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:19.640992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0e42e-9e98-4a53-9944-e7665e55c323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:19.641370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0e42e-9e98-4a53-9944-e7665e55c323 admin admin] PATH_INFO: `/v3/projects/d9a60de4a5a042149f7819eb10552a43` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:19.642610 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0e42e-9e98-4a53-9944-e7665e55c323 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9a60de4a5a042149f7819eb10552a43)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:19.644616 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0e42e-9e98-4a53-9944-e7665e55c323 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:19.694126 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5af0e42e-9e98-4a53-9944-e7665e55c323 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9a60de4a5a042149f7819eb10552a43', 'request_id': 'req-5af0e42e-9e98-4a53-9944-e7665e55c323'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:19.701674 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5af0e42e-9e98-4a53-9944-e7665e55c323 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9a60de4a5a042149f7819eb10552a43', 'request_id': 'req-5af0e42e-9e98-4a53-9944-e7665e55c323'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:19.707293 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5af0e42e-9e98-4a53-9944-e7665e55c323 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9a60de4a5a042149f7819eb10552a43', 'request_id': 'req-5af0e42e-9e98-4a53-9944-e7665e55c323'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:19.718879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5af0e42e-9e98-4a53-9944-e7665e55c323 admin admin] The token cache is being invalidate because project d9a60de4a5a042149f7819eb10552a43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:21:19.719097 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5af0e42e-9e98-4a53-9944-e7665e55c323 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:19.719756 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-5af0e42e-9e98-4a53-9944-e7665e55c323 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:21:19.721320 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1370/2721] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:21:19 2022] DELETE /identity/v3/projects/d9a60de4a5a042149f7819eb10552a43 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:21:19.726280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9697b-e764-4bd2-8ed1-1453f8c192da None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:19.726838 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9697b-e764-4bd2-8ed1-1453f8c192da None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:19.726838 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9697b-e764-4bd2-8ed1-1453f8c192da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:19.749843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-bde9697b-e764-4bd2-8ed1-1453f8c192da None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:19.751059 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-bde9697b-e764-4bd2-8ed1-1453f8c192da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:19.770979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:19.817379 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1352/2722] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:19.827650 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2479e9-9c8b-4bad-a20e-3891e350869c None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:19.828146 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2479e9-9c8b-4bad-a20e-3891e350869c None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:19.828244 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2479e9-9c8b-4bad-a20e-3891e350869c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:19.975485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-3d2479e9-9c8b-4bad-a20e-3891e350869c None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:19.976704 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3d2479e9-9c8b-4bad-a20e-3891e350869c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:19.984826 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1371/2723] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:19.989186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9697b-e764-4bd2-8ed1-1453f8c192da None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:19.999126 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9697b-e764-4bd2-8ed1-1453f8c192da None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:20.003873 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c9563-694c-4790-a8a4-5370c31c5fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:20.004577 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c9563-694c-4790-a8a4-5370c31c5fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:20.004860 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c9563-694c-4790-a8a4-5370c31c5fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:20.005145 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c9563-694c-4790-a8a4-5370c31c5fa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:20.005835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c9563-694c-4790-a8a4-5370c31c5fa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:21:20.006133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c9563-694c-4790-a8a4-5370c31c5fa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:20.007747 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c9563-694c-4790-a8a4-5370c31c5fa1 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:20.020088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1353/2724] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:20.024561 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2479e9-9c8b-4bad-a20e-3891e350869c None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:20.034743 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2479e9-9c8b-4bad-a20e-3891e350869c None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:20.039444 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8178c-7c3c-4a4c-a72c-64f238eb7bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:20.040050 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8178c-7c3c-4a4c-a72c-64f238eb7bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:20.040329 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8178c-7c3c-4a4c-a72c-64f238eb7bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:20.040639 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8178c-7c3c-4a4c-a72c-64f238eb7bd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:20.041287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b8178c-7c3c-4a4c-a72c-64f238eb7bd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:20.043169 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b8178c-7c3c-4a4c-a72c-64f238eb7bd5 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:20.072879 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1372/2725] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:21:20 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:21:20.077709 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c9563-694c-4790-a8a4-5370c31c5fa1 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:20.088565 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c9563-694c-4790-a8a4-5370c31c5fa1 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:20.146351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607a650-8b06-4c82-9dfb-2d41d8172365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:20.146966 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607a650-8b06-4c82-9dfb-2d41d8172365 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:20.147247 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607a650-8b06-4c82-9dfb-2d41d8172365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:20.147520 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607a650-8b06-4c82-9dfb-2d41d8172365 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:20.148181 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607a650-8b06-4c82-9dfb-2d41d8172365 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:20.150148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607a650-8b06-4c82-9dfb-2d41d8172365 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:20.172363 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1354/2726] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:21:20 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:21:20.176528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8178c-7c3c-4a4c-a72c-64f238eb7bd5 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:20.187001 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8178c-7c3c-4a4c-a72c-64f238eb7bd5 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:20.191703 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fba5a4-cea3-47cc-aa24-91b6a474a3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:20.192359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fba5a4-cea3-47cc-aa24-91b6a474a3ae admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:20.192675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fba5a4-cea3-47cc-aa24-91b6a474a3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:20.192947 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fba5a4-cea3-47cc-aa24-91b6a474a3ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:20.193640 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fba5a4-cea3-47cc-aa24-91b6a474a3ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:20.195240 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fba5a4-cea3-47cc-aa24-91b6a474a3ae admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:20.205795 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1373/2727] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:21:20.209930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607a650-8b06-4c82-9dfb-2d41d8172365 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:20.221686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607a650-8b06-4c82-9dfb-2d41d8172365 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:20.226697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fffc9d8-6437-46ec-93ad-19e6fa703cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:20.227230 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffc9d8-6437-46ec-93ad-19e6fa703cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:20.227516 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffc9d8-6437-46ec-93ad-19e6fa703cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:20.228064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffc9d8-6437-46ec-93ad-19e6fa703cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:20.228783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fffc9d8-6437-46ec-93ad-19e6fa703cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:20.230384 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fffc9d8-6437-46ec-93ad-19e6fa703cd2 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:20.245129 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1355/2728] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:21:20.250605 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fba5a4-cea3-47cc-aa24-91b6a474a3ae admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:20.261088 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fba5a4-cea3-47cc-aa24-91b6a474a3ae admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:20.266707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3b325-16a2-4bba-8bf9-0e5e30cdf6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:20.267366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3b325-16a2-4bba-8bf9-0e5e30cdf6d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:20.267642 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3b325-16a2-4bba-8bf9-0e5e30cdf6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:20.267911 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3b325-16a2-4bba-8bf9-0e5e30cdf6d1 admin admin] PATH_INFO: `/v3/projects/4fb174cae05a47ab901292a7d516f2bb/users/b6a5b96d33aa4b7a8e50fa27925f4b0d/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:20.283299 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3b325-16a2-4bba-8bf9-0e5e30cdf6d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fb174cae05a47ab901292a7d516f2bb, user_id=b6a5b96d33aa4b7a8e50fa27925f4b0d, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:20.286017 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3b325-16a2-4bba-8bf9-0e5e30cdf6d1 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:20.299700 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1374/2729] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:21:20 2022] PUT /identity/v3/projects/4fb174cae05a47ab901292a7d516f2bb/users/b6a5b96d33aa4b7a8e50fa27925f4b0d/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:21:20.305110 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d062e-7a4b-4ff1-9e8b-b6bbfc4ad9f1 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:20.305459 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d062e-7a4b-4ff1-9e8b-b6bbfc4ad9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:20.305735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d062e-7a4b-4ff1-9e8b-b6bbfc4ad9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:20.325510 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-244d062e-7a4b-4ff1-9e8b-b6bbfc4ad9f1 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:20.326735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-244d062e-7a4b-4ff1-9e8b-b6bbfc4ad9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:20.391530 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1356/2730] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:20.401783 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3b325-16a2-4bba-8bf9-0e5e30cdf6d1 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:20.403721 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a0e3b325-16a2-4bba-8bf9-0e5e30cdf6d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:20.425895 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:20.465931 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3b325-16a2-4bba-8bf9-0e5e30cdf6d1 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:20.469774 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e51b2e-b957-4400-8ffb-4a27812aed4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:20.470385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e51b2e-b957-4400-8ffb-4a27812aed4f service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:20.470659 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e51b2e-b957-4400-8ffb-4a27812aed4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:20.470929 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e51b2e-b957-4400-8ffb-4a27812aed4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:20.480762 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e51b2e-b957-4400-8ffb-4a27812aed4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:20.482114 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e51b2e-b957-4400-8ffb-4a27812aed4f service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:20.492524 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1375/2731] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:21:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:21:21.226851 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16340510-e005-44fd-956d-fe240d07c5fb None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:21.227819 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16340510-e005-44fd-956d-fe240d07c5fb None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:21.227819 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16340510-e005-44fd-956d-fe240d07c5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:21.258586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-16340510-e005-44fd-956d-fe240d07c5fb None None] MFA Rules not processed for user `b6a5b96d33aa4b7a8e50fa27925f4b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:21.259901 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-16340510-e005-44fd-956d-fe240d07c5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:21.279249 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:21.343476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1357/2732] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:21:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:21.355811 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e51b2e-b957-4400-8ffb-4a27812aed4f service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:21.357537 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-01e51b2e-b957-4400-8ffb-4a27812aed4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:21.420758 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e51b2e-b957-4400-8ffb-4a27812aed4f service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:21.424310 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1092d-7bec-4fea-87bc-f628c709f237 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:21.424965 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1092d-7bec-4fea-87bc-f628c709f237 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:21.425239 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1092d-7bec-4fea-87bc-f628c709f237 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:21.425511 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1092d-7bec-4fea-87bc-f628c709f237 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:21.434185 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1092d-7bec-4fea-87bc-f628c709f237 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:21.435600 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1092d-7bec-4fea-87bc-f628c709f237 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:21.445176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1376/2733] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:21:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:21:22.822706 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabac8da-46b2-4a51-aee2-b9c18ed1d238 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:22.823259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabac8da-46b2-4a51-aee2-b9c18ed1d238 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:22.823259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabac8da-46b2-4a51-aee2-b9c18ed1d238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:22.843459 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-eabac8da-46b2-4a51-aee2-b9c18ed1d238 None None] MFA Rules not processed for user `b6a5b96d33aa4b7a8e50fa27925f4b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:22.844706 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-eabac8da-46b2-4a51-aee2-b9c18ed1d238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:22.853220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1358/2734] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:21:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:31.440915 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=43 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:32.837812 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=22 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:38.961365 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1092d-7bec-4fea-87bc-f628c709f237 service cinder] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:38.972574 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1092d-7bec-4fea-87bc-f628c709f237 service cinder] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:38.976171 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e7735-d5ae-46ae-9c72-f6c984237e1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:38.976813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e7735-d5ae-46ae-9c72-f6c984237e1b service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:38.977085 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e7735-d5ae-46ae-9c72-f6c984237e1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:38.977352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e7735-d5ae-46ae-9c72-f6c984237e1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:38.979338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-234e7735-d5ae-46ae-9c72-f6c984237e1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:38.990684 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e7735-d5ae-46ae-9c72-f6c984237e1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:38.992390 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e7735-d5ae-46ae-9c72-f6c984237e1b service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:39.003404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1377/2735] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:21:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:21:48.998744 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=6 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:52.064729 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabac8da-46b2-4a51-aee2-b9c18ed1d238 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:52.076604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabac8da-46b2-4a51-aee2-b9c18ed1d238 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:52.081190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b39881-92a3-4137-a573-fb8b3418df01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:52.081620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b39881-92a3-4137-a573-fb8b3418df01 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:52.081908 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b39881-92a3-4137-a573-fb8b3418df01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:52.082178 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b39881-92a3-4137-a573-fb8b3418df01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:52.083951 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-42b39881-92a3-4137-a573-fb8b3418df01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:52.146574 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b39881-92a3-4137-a573-fb8b3418df01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:52.148644 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b39881-92a3-4137-a573-fb8b3418df01 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:52.158341 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1359/2736] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:21:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:21:54.970675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f607495-66e7-4630-9553-f7a11fcf7e89 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:54.971200 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f607495-66e7-4630-9553-f7a11fcf7e89 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:54.971200 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f607495-66e7-4630-9553-f7a11fcf7e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:54.990798 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-1f607495-66e7-4630-9553-f7a11fcf7e89 None None] MFA Rules not processed for user `9673eb00ccc24fbaa495ddcbaf6d3076`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:54.992004 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-1f607495-66e7-4630-9553-f7a11fcf7e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:55.046898 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1378/2737] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:21:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:22:00.193447 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bfb17-5c2b-4f46-95ce-2b589cabc63f None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:00.193925 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bfb17-5c2b-4f46-95ce-2b589cabc63f None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:00.194025 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bfb17-5c2b-4f46-95ce-2b589cabc63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:00.214508 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-372bfb17-5c2b-4f46-95ce-2b589cabc63f None None] MFA Rules not processed for user `b687d38492594eab876740bdfb7db9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:22:00.215713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-372bfb17-5c2b-4f46-95ce-2b589cabc63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:00.223481 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1360/2738] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:22:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:22:05.039881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=25 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:10.208236 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=31 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:35.483324 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e63f5-256d-42af-bd24-8c0cef22c854 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:35.484110 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e63f5-256d-42af-bd24-8c0cef22c854 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:35.484110 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e63f5-256d-42af-bd24-8c0cef22c854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:35.503759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-181e63f5-256d-42af-bd24-8c0cef22c854 None None] MFA Rules not processed for user `b6a5b96d33aa4b7a8e50fa27925f4b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:22:35.505014 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-181e63f5-256d-42af-bd24-8c0cef22c854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:35.512393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1379/2739] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:22:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:22:38.382343 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bfb17-5c2b-4f46-95ce-2b589cabc63f None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:38.394200 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bfb17-5c2b-4f46-95ce-2b589cabc63f None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:38.398439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12884c1-b1ff-4536-8024-dcf8589509f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:38.399079 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12884c1-b1ff-4536-8024-dcf8589509f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:38.399357 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12884c1-b1ff-4536-8024-dcf8589509f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:38.399798 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12884c1-b1ff-4536-8024-dcf8589509f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:38.402152 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-f12884c1-b1ff-4536-8024-dcf8589509f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:38.514593 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12884c1-b1ff-4536-8024-dcf8589509f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:38.516037 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12884c1-b1ff-4536-8024-dcf8589509f5 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:38.529677 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1361/2740] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:22:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:22:45.497964 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:46.895221 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e63f5-256d-42af-bd24-8c0cef22c854 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:46.896831 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-181e63f5-256d-42af-bd24-8c0cef22c854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:46.911425 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e63f5-256d-42af-bd24-8c0cef22c854 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:46.916897 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fa5d4-ecba-47b6-9f5a-ae74c8e73a65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:46.917652 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fa5d4-ecba-47b6-9f5a-ae74c8e73a65 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:46.917967 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fa5d4-ecba-47b6-9f5a-ae74c8e73a65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:46.918449 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fa5d4-ecba-47b6-9f5a-ae74c8e73a65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:46.920620 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ec5fa5d4-ecba-47b6-9f5a-ae74c8e73a65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:46.938392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5fa5d4-ecba-47b6-9f5a-ae74c8e73a65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:46.939289 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5fa5d4-ecba-47b6-9f5a-ae74c8e73a65 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:46.953472 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1380/2741] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:22:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:22:48.524066 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=34 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:56.946543 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=6 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:16.988329 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd606865-df36-4d01-903e-a24319892eaa None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:16.988999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd606865-df36-4d01-903e-a24319892eaa None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:16.988999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd606865-df36-4d01-903e-a24319892eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:17.010717 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-fd606865-df36-4d01-903e-a24319892eaa None None] MFA Rules not processed for user `9673eb00ccc24fbaa495ddcbaf6d3076`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:17.012624 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-fd606865-df36-4d01-903e-a24319892eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:17.020505 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1362/2742] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:23:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:27.003726 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:28.383022 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fa5d4-ecba-47b6-9f5a-ae74c8e73a65 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:28.397264 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fa5d4-ecba-47b6-9f5a-ae74c8e73a65 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:28.403280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ca825-9617-4ef0-a066-62a77d254aae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:28.404284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ca825-9617-4ef0-a066-62a77d254aae service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:28.404668 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ca825-9617-4ef0-a066-62a77d254aae service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:28.405977 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ca825-9617-4ef0-a066-62a77d254aae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:28.409450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-c11ca825-9617-4ef0-a066-62a77d254aae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:28.421475 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ca825-9617-4ef0-a066-62a77d254aae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:28.422938 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ca825-9617-4ef0-a066-62a77d254aae service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:28.443453 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1381/2743] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:32.836265 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd606865-df36-4d01-903e-a24319892eaa None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:32.837805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-fd606865-df36-4d01-903e-a24319892eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:32.855724 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd606865-df36-4d01-903e-a24319892eaa None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:32.861878 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dca543-62b4-4a0e-9437-912b8639ce78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:32.862616 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dca543-62b4-4a0e-9437-912b8639ce78 service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:32.862965 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dca543-62b4-4a0e-9437-912b8639ce78 service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:32.863299 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dca543-62b4-4a0e-9437-912b8639ce78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:32.865315 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-40dca543-62b4-4a0e-9437-912b8639ce78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:32.879152 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dca543-62b4-4a0e-9437-912b8639ce78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:32.880900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dca543-62b4-4a0e-9437-912b8639ce78 service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:32.892849 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1363/2744] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:37.316122 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ca825-9617-4ef0-a066-62a77d254aae service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:37.335081 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ca825-9617-4ef0-a066-62a77d254aae service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:37.347819 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23358658-a22f-4c2e-b0f8-339bc6514f49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:37.349394 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23358658-a22f-4c2e-b0f8-339bc6514f49 service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:37.349759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23358658-a22f-4c2e-b0f8-339bc6514f49 service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:37.350330 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23358658-a22f-4c2e-b0f8-339bc6514f49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:37.353438 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-23358658-a22f-4c2e-b0f8-339bc6514f49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:37.390255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23358658-a22f-4c2e-b0f8-339bc6514f49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:37.391887 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23358658-a22f-4c2e-b0f8-339bc6514f49 service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:37.406110 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1382/2745] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:23:37.651319 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dca543-62b4-4a0e-9437-912b8639ce78 service nova] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:37.654533 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-40dca543-62b4-4a0e-9437-912b8639ce78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:37.731794 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23358658-a22f-4c2e-b0f8-339bc6514f49 service nova] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:37.755324 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23358658-a22f-4c2e-b0f8-339bc6514f49 service nova] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:37.776862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6ca21-fe89-4151-99aa-9d2c831670ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:37.778160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6ca21-fe89-4151-99aa-9d2c831670ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:37.778750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6ca21-fe89-4151-99aa-9d2c831670ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:37.779556 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6ca21-fe89-4151-99aa-9d2c831670ce admin admin] PATH_INFO: `/v3/users/b6a5b96d33aa4b7a8e50fa27925f4b0d` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:37.782485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6ca21-fe89-4151-99aa-9d2c831670ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6a5b96d33aa4b7a8e50fa27925f4b0d)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:37.789269 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6ca21-fe89-4151-99aa-9d2c831670ce admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:37.841909 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dca543-62b4-4a0e-9437-912b8639ce78 service nova] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:37.847995 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619d156-7b21-4183-a0d7-e3671867b93e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07e3e25e6a8466fb237dcb836aecda2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:37.848664 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619d156-7b21-4183-a0d7-e3671867b93e service placement] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:37.849004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619d156-7b21-4183-a0d7-e3671867b93e service placement] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:37.849307 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619d156-7b21-4183-a0d7-e3671867b93e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:37.851453 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-f619d156-7b21-4183-a0d7-e3671867b93e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:37.865450 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619d156-7b21-4183-a0d7-e3671867b93e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:37.867125 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619d156-7b21-4183-a0d7-e3671867b93e service placement] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:37.879682 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a3c6ca21-fe89-4151-99aa-9d2c831670ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6a5b96d33aa4b7a8e50fa27925f4b0d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:37.880173 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1364/2746] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:37.890517 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a3c6ca21-fe89-4151-99aa-9d2c831670ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6a5b96d33aa4b7a8e50fa27925f4b0d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:37.900913 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a3c6ca21-fe89-4151-99aa-9d2c831670ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6a5b96d33aa4b7a8e50fa27925f4b0d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:37.903206 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a3c6ca21-fe89-4151-99aa-9d2c831670ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6a5b96d33aa4b7a8e50fa27925f4b0d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:37.904792 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-a3c6ca21-fe89-4151-99aa-9d2c831670ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6a5b96d33aa4b7a8e50fa27925f4b0d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:37.937408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1383/2747] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:23:37 2022] DELETE /identity/v3/users/b6a5b96d33aa4b7a8e50fa27925f4b0d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:38.148369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619d156-7b21-4183-a0d7-e3671867b93e service placement] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:38.160959 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619d156-7b21-4183-a0d7-e3671867b93e service placement] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:38.239223 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e889b-56cb-4f50-b37f-99792e62f6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:38.239979 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e889b-56cb-4f50-b37f-99792e62f6e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:38.240263 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e889b-56cb-4f50-b37f-99792e62f6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:38.240579 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e889b-56cb-4f50-b37f-99792e62f6e6 admin admin] PATH_INFO: `/v3/projects/4fb174cae05a47ab901292a7d516f2bb` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:38.241804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441e889b-56cb-4f50-b37f-99792e62f6e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fb174cae05a47ab901292a7d516f2bb)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:38.243712 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441e889b-56cb-4f50-b37f-99792e62f6e6 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:38.291538 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-441e889b-56cb-4f50-b37f-99792e62f6e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fb174cae05a47ab901292a7d516f2bb', 'request_id': 'req-441e889b-56cb-4f50-b37f-99792e62f6e6'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:38.298675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-441e889b-56cb-4f50-b37f-99792e62f6e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fb174cae05a47ab901292a7d516f2bb', 'request_id': 'req-441e889b-56cb-4f50-b37f-99792e62f6e6'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:38.304235 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-441e889b-56cb-4f50-b37f-99792e62f6e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fb174cae05a47ab901292a7d516f2bb', 'request_id': 'req-441e889b-56cb-4f50-b37f-99792e62f6e6'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:38.316759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-441e889b-56cb-4f50-b37f-99792e62f6e6 admin admin] The token cache is being invalidate because project 4fb174cae05a47ab901292a7d516f2bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:23:38.317045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-441e889b-56cb-4f50-b37f-99792e62f6e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:38.317725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-441e889b-56cb-4f50-b37f-99792e62f6e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:38.319349 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1365/2748] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:23:38 2022] DELETE /identity/v3/projects/4fb174cae05a47ab901292a7d516f2bb => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:38.709460 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6ca21-fe89-4151-99aa-9d2c831670ce admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:38.710963 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a3c6ca21-fe89-4151-99aa-9d2c831670ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:38.772389 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6ca21-fe89-4151-99aa-9d2c831670ce admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:38.823040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2bf772-b38c-45e0-a2a1-71b2ebea309f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:38.824179 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bf772-b38c-45e0-a2a1-71b2ebea309f service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:38.824453 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bf772-b38c-45e0-a2a1-71b2ebea309f service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:38.824751 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bf772-b38c-45e0-a2a1-71b2ebea309f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:38.826773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-9c2bf772-b38c-45e0-a2a1-71b2ebea309f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:38.848302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:38.901414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2bf772-b38c-45e0-a2a1-71b2ebea309f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:38.902855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2bf772-b38c-45e0-a2a1-71b2ebea309f service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:38.912953 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1384/2749] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:46.784138 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e889b-56cb-4f50-b37f-99792e62f6e6 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:46.786065 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-441e889b-56cb-4f50-b37f-99792e62f6e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:46.802467 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e889b-56cb-4f50-b37f-99792e62f6e6 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:46.810216 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52af115-d009-4f06-b535-4381bc70e0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:46.810872 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52af115-d009-4f06-b535-4381bc70e0fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:46.811159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52af115-d009-4f06-b535-4381bc70e0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:46.811575 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52af115-d009-4f06-b535-4381bc70e0fd admin admin] PATH_INFO: `/v3/users/31789e13f7824dae998c99eefc7aaaaf` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:46.812758 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52af115-d009-4f06-b535-4381bc70e0fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=31789e13f7824dae998c99eefc7aaaaf)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:46.814706 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52af115-d009-4f06-b535-4381bc70e0fd admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:46.816510 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:46.884326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c52af115-d009-4f06-b535-4381bc70e0fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31789e13f7824dae998c99eefc7aaaaf'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:46.884964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c52af115-d009-4f06-b535-4381bc70e0fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31789e13f7824dae998c99eefc7aaaaf'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:46.893990 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c52af115-d009-4f06-b535-4381bc70e0fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31789e13f7824dae998c99eefc7aaaaf'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:46.894612 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c52af115-d009-4f06-b535-4381bc70e0fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31789e13f7824dae998c99eefc7aaaaf'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:46.909176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c52af115-d009-4f06-b535-4381bc70e0fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31789e13f7824dae998c99eefc7aaaaf'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:46.922989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1366/2750] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:23:46 2022] DELETE /identity/v3/users/31789e13f7824dae998c99eefc7aaaaf => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:47.163231 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2bf772-b38c-45e0-a2a1-71b2ebea309f service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:47.177745 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2bf772-b38c-45e0-a2a1-71b2ebea309f service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:47.296895 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e22c8b-b097-41f0-a357-45871dec1459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:47.297736 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e22c8b-b097-41f0-a357-45871dec1459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:47.298163 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e22c8b-b097-41f0-a357-45871dec1459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:47.298557 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e22c8b-b097-41f0-a357-45871dec1459 admin admin] PATH_INFO: `/v3/projects/0dbf7f1737834b12a093b137169033c5` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:47.300500 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e22c8b-b097-41f0-a357-45871dec1459 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dbf7f1737834b12a093b137169033c5)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:47.303370 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e22c8b-b097-41f0-a357-45871dec1459 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:47.387238 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-92e22c8b-b097-41f0-a357-45871dec1459 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dbf7f1737834b12a093b137169033c5', 'request_id': 'req-92e22c8b-b097-41f0-a357-45871dec1459'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:47.400292 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-92e22c8b-b097-41f0-a357-45871dec1459 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dbf7f1737834b12a093b137169033c5', 'request_id': 'req-92e22c8b-b097-41f0-a357-45871dec1459'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:47.411743 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-92e22c8b-b097-41f0-a357-45871dec1459 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dbf7f1737834b12a093b137169033c5', 'request_id': 'req-92e22c8b-b097-41f0-a357-45871dec1459'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:47.435397 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-92e22c8b-b097-41f0-a357-45871dec1459 admin admin] The token cache is being invalidate because project 0dbf7f1737834b12a093b137169033c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:23:47.435755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-92e22c8b-b097-41f0-a357-45871dec1459 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:47.436907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-92e22c8b-b097-41f0-a357-45871dec1459 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:47.439028 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1385/2751] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:23:47 2022] DELETE /identity/v3/projects/0dbf7f1737834b12a093b137169033c5 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:47.443935 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52af115-d009-4f06-b535-4381bc70e0fd admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:47.446589 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c52af115-d009-4f06-b535-4381bc70e0fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:47.472614 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52af115-d009-4f06-b535-4381bc70e0fd admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:47.557127 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd705f-7283-4074-979b-1ba4205e3ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:47.557822 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd705f-7283-4074-979b-1ba4205e3ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:47.558326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd705f-7283-4074-979b-1ba4205e3ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:47.559270 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd705f-7283-4074-979b-1ba4205e3ce8 admin admin] PATH_INFO: `/v3/users/8c976d7293bc495a82ce33404c638103` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:47.560941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd705f-7283-4074-979b-1ba4205e3ce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c976d7293bc495a82ce33404c638103)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:47.563011 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd705f-7283-4074-979b-1ba4205e3ce8 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:47.622086 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-85fd705f-7283-4074-979b-1ba4205e3ce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c976d7293bc495a82ce33404c638103'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:47.622739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-85fd705f-7283-4074-979b-1ba4205e3ce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c976d7293bc495a82ce33404c638103'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:47.633508 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-85fd705f-7283-4074-979b-1ba4205e3ce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c976d7293bc495a82ce33404c638103'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:47.634176 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-85fd705f-7283-4074-979b-1ba4205e3ce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c976d7293bc495a82ce33404c638103'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:47.651002 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-85fd705f-7283-4074-979b-1ba4205e3ce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c976d7293bc495a82ce33404c638103'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:47.667402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:47.676938 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1367/2752] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:23:47 2022] DELETE /identity/v3/users/8c976d7293bc495a82ce33404c638103 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:47.954980 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e22c8b-b097-41f0-a357-45871dec1459 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:47.956725 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-92e22c8b-b097-41f0-a357-45871dec1459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:47.972602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e22c8b-b097-41f0-a357-45871dec1459 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:48.028496 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:48.054942 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f85e7-0968-4d91-a4f8-7f0925418e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:48.055614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f85e7-0968-4d91-a4f8-7f0925418e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:48.055888 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f85e7-0968-4d91-a4f8-7f0925418e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:48.056163 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f85e7-0968-4d91-a4f8-7f0925418e42 admin admin] PATH_INFO: `/v3/projects/a4067667ba2e4051bc27bb45bd7b5bad` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:48.057910 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f85e7-0968-4d91-a4f8-7f0925418e42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4067667ba2e4051bc27bb45bd7b5bad)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:48.059865 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f85e7-0968-4d91-a4f8-7f0925418e42 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:48.114630 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ee8f85e7-0968-4d91-a4f8-7f0925418e42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4067667ba2e4051bc27bb45bd7b5bad', 'request_id': 'req-ee8f85e7-0968-4d91-a4f8-7f0925418e42'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:48.124238 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ee8f85e7-0968-4d91-a4f8-7f0925418e42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4067667ba2e4051bc27bb45bd7b5bad', 'request_id': 'req-ee8f85e7-0968-4d91-a4f8-7f0925418e42'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:48.131722 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ee8f85e7-0968-4d91-a4f8-7f0925418e42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4067667ba2e4051bc27bb45bd7b5bad', 'request_id': 'req-ee8f85e7-0968-4d91-a4f8-7f0925418e42'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:48.150802 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ee8f85e7-0968-4d91-a4f8-7f0925418e42 admin admin] The token cache is being invalidate because project a4067667ba2e4051bc27bb45bd7b5bad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:23:48.151705 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ee8f85e7-0968-4d91-a4f8-7f0925418e42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:48.152503 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-ee8f85e7-0968-4d91-a4f8-7f0925418e42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:48.154204 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1386/2753] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:23:47 2022] DELETE /identity/v3/projects/a4067667ba2e4051bc27bb45bd7b5bad => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:57.683392 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:58.146792 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=32,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:28.241645 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65860c-bbd9-4cfb-a62f-ded7ec606147 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:28.242903 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65860c-bbd9-4cfb-a62f-ded7ec606147 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:28.243185 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65860c-bbd9-4cfb-a62f-ded7ec606147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:28.263144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-0d65860c-bbd9-4cfb-a62f-ded7ec606147 None None] MFA Rules not processed for user `9673eb00ccc24fbaa495ddcbaf6d3076`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:28.264368 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-0d65860c-bbd9-4cfb-a62f-ded7ec606147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:28.323312 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1368/2754] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:24:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:29.008353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f85e7-0968-4d91-a4f8-7f0925418e42 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:29.009986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ee8f85e7-0968-4d91-a4f8-7f0925418e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:29.096396 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f85e7-0968-4d91-a4f8-7f0925418e42 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:29.101847 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697ebb6-13c4-435b-bda8-be91baad68e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07e3e25e6a8466fb237dcb836aecda2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:29.102485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697ebb6-13c4-435b-bda8-be91baad68e3 service placement] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:29.102764 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697ebb6-13c4-435b-bda8-be91baad68e3 service placement] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:29.103039 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697ebb6-13c4-435b-bda8-be91baad68e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:29.104918 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0697ebb6-13c4-435b-bda8-be91baad68e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:29.117233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0697ebb6-13c4-435b-bda8-be91baad68e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:29.118831 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0697ebb6-13c4-435b-bda8-be91baad68e3 service placement] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:29.131746 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1387/2755] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:32.808182 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65860c-bbd9-4cfb-a62f-ded7ec606147 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:32.809626 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-0d65860c-bbd9-4cfb-a62f-ded7ec606147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:32.868489 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65860c-bbd9-4cfb-a62f-ded7ec606147 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:32.872567 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fab1ec-349d-4de5-aded-8d7bd4f440b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:32.873180 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fab1ec-349d-4de5-aded-8d7bd4f440b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:32.873455 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fab1ec-349d-4de5-aded-8d7bd4f440b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:32.873722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fab1ec-349d-4de5-aded-8d7bd4f440b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:32.875466 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d0fab1ec-349d-4de5-aded-8d7bd4f440b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:32.940701 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fab1ec-349d-4de5-aded-8d7bd4f440b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:32.942134 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fab1ec-349d-4de5-aded-8d7bd4f440b5 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:32.951804 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1369/2756] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:36.715501 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697ebb6-13c4-435b-bda8-be91baad68e3 service placement] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:36.717614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0697ebb6-13c4-435b-bda8-be91baad68e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:36.824544 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697ebb6-13c4-435b-bda8-be91baad68e3 service placement] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:36.830162 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17697de0-7eca-44c7-8441-baf8b9129858 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:36.830732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17697de0-7eca-44c7-8441-baf8b9129858 service nova] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:36.831011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17697de0-7eca-44c7-8441-baf8b9129858 service nova] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:36.831281 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17697de0-7eca-44c7-8441-baf8b9129858 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:36.837287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-17697de0-7eca-44c7-8441-baf8b9129858 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:36.851009 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17697de0-7eca-44c7-8441-baf8b9129858 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:36.852863 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17697de0-7eca-44c7-8441-baf8b9129858 service nova] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:36.867528 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1388/2757] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:40.027401 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fab1ec-349d-4de5-aded-8d7bd4f440b5 service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:40.030193 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-d0fab1ec-349d-4de5-aded-8d7bd4f440b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:40.049431 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fab1ec-349d-4de5-aded-8d7bd4f440b5 service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:40.055401 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250802e9-348d-4e78-97d1-2e221f64b8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:40.056133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250802e9-348d-4e78-97d1-2e221f64b8ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:40.056400 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250802e9-348d-4e78-97d1-2e221f64b8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:40.056707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250802e9-348d-4e78-97d1-2e221f64b8ec admin admin] PATH_INFO: `/v3/users/b687d38492594eab876740bdfb7db9ff` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:40.057937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250802e9-348d-4e78-97d1-2e221f64b8ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=b687d38492594eab876740bdfb7db9ff)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:40.059870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250802e9-348d-4e78-97d1-2e221f64b8ec admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:40.118882 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-250802e9-348d-4e78-97d1-2e221f64b8ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b687d38492594eab876740bdfb7db9ff'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:40.119564 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-250802e9-348d-4e78-97d1-2e221f64b8ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b687d38492594eab876740bdfb7db9ff'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:40.122581 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:40.130213 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-250802e9-348d-4e78-97d1-2e221f64b8ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b687d38492594eab876740bdfb7db9ff'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:40.131788 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-250802e9-348d-4e78-97d1-2e221f64b8ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b687d38492594eab876740bdfb7db9ff'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:40.147226 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-250802e9-348d-4e78-97d1-2e221f64b8ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b687d38492594eab876740bdfb7db9ff'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:40.161895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1370/2758] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:24:40 2022] DELETE /identity/v3/users/b687d38492594eab876740bdfb7db9ff => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:40.358857 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17697de0-7eca-44c7-8441-baf8b9129858 service nova] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:40.373903 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17697de0-7eca-44c7-8441-baf8b9129858 service nova] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:40.441773 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e724e94-40b1-464c-b2c7-df0f3e08c82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:40.442420 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e724e94-40b1-464c-b2c7-df0f3e08c82a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:40.442703 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e724e94-40b1-464c-b2c7-df0f3e08c82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:40.443411 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e724e94-40b1-464c-b2c7-df0f3e08c82a admin admin] PATH_INFO: `/v3/projects/821ce227288a424ea530e4a7c08e41c9` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:40.444317 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e724e94-40b1-464c-b2c7-df0f3e08c82a admin admin] RBAC: Authorizing `identity:delete_project(project_id=821ce227288a424ea530e4a7c08e41c9)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:40.446273 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e724e94-40b1-464c-b2c7-df0f3e08c82a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:40.463797 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:40.505662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-8e724e94-40b1-464c-b2c7-df0f3e08c82a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '821ce227288a424ea530e4a7c08e41c9', 'request_id': 'req-8e724e94-40b1-464c-b2c7-df0f3e08c82a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:40.518545 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-8e724e94-40b1-464c-b2c7-df0f3e08c82a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '821ce227288a424ea530e4a7c08e41c9', 'request_id': 'req-8e724e94-40b1-464c-b2c7-df0f3e08c82a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:40.527105 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-8e724e94-40b1-464c-b2c7-df0f3e08c82a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '821ce227288a424ea530e4a7c08e41c9', 'request_id': 'req-8e724e94-40b1-464c-b2c7-df0f3e08c82a'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:40.553161 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-8e724e94-40b1-464c-b2c7-df0f3e08c82a admin admin] The token cache is being invalidate because project 821ce227288a424ea530e4a7c08e41c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:40.553618 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-8e724e94-40b1-464c-b2c7-df0f3e08c82a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:40.554663 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-8e724e94-40b1-464c-b2c7-df0f3e08c82a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:40.556334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1389/2759] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:40 2022] DELETE /identity/v3/projects/821ce227288a424ea530e4a7c08e41c9 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:40.560774 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250802e9-348d-4e78-97d1-2e221f64b8ec admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:40.562311 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-250802e9-348d-4e78-97d1-2e221f64b8ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:40.577405 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250802e9-348d-4e78-97d1-2e221f64b8ec admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:40.669083 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2d9b8-5589-4005-9798-14a90b2b83a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:40.670665 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2d9b8-5589-4005-9798-14a90b2b83a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:40.670665 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2d9b8-5589-4005-9798-14a90b2b83a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:40.670665 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2d9b8-5589-4005-9798-14a90b2b83a7 admin admin] PATH_INFO: `/v3/users/1b2f63b200cc4208bd8df3b407eb8904` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:40.671919 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2d9b8-5589-4005-9798-14a90b2b83a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b2f63b200cc4208bd8df3b407eb8904)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:40.674739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2d9b8-5589-4005-9798-14a90b2b83a7 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:40.732713 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-cad2d9b8-5589-4005-9798-14a90b2b83a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b2f63b200cc4208bd8df3b407eb8904'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:40.733605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-cad2d9b8-5589-4005-9798-14a90b2b83a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b2f63b200cc4208bd8df3b407eb8904'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:40.741864 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-cad2d9b8-5589-4005-9798-14a90b2b83a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b2f63b200cc4208bd8df3b407eb8904'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:40.742656 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-cad2d9b8-5589-4005-9798-14a90b2b83a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b2f63b200cc4208bd8df3b407eb8904'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:40.759030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-cad2d9b8-5589-4005-9798-14a90b2b83a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b2f63b200cc4208bd8df3b407eb8904'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:40.783347 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1371/2760] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:24:40 2022] DELETE /identity/v3/users/1b2f63b200cc4208bd8df3b407eb8904 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:40.976247 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e724e94-40b1-464c-b2c7-df0f3e08c82a admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:40.978652 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-8e724e94-40b1-464c-b2c7-df0f3e08c82a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:40.994734 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e724e94-40b1-464c-b2c7-df0f3e08c82a admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:41.096987 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08db58-959b-4835-b620-b4df89ede70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:41.098228 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08db58-959b-4835-b620-b4df89ede70b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:41.098981 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08db58-959b-4835-b620-b4df89ede70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:41.099481 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08db58-959b-4835-b620-b4df89ede70b admin admin] PATH_INFO: `/v3/projects/5c98a85ab7ea46189515430e8dc1b492` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:41.101950 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08db58-959b-4835-b620-b4df89ede70b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c98a85ab7ea46189515430e8dc1b492)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:41.105842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08db58-959b-4835-b620-b4df89ede70b admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:41.195853 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-bd08db58-959b-4835-b620-b4df89ede70b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c98a85ab7ea46189515430e8dc1b492', 'request_id': 'req-bd08db58-959b-4835-b620-b4df89ede70b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:41.207549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-bd08db58-959b-4835-b620-b4df89ede70b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c98a85ab7ea46189515430e8dc1b492', 'request_id': 'req-bd08db58-959b-4835-b620-b4df89ede70b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:41.219189 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-bd08db58-959b-4835-b620-b4df89ede70b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c98a85ab7ea46189515430e8dc1b492', 'request_id': 'req-bd08db58-959b-4835-b620-b4df89ede70b'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:41.240494 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-bd08db58-959b-4835-b620-b4df89ede70b admin admin] The token cache is being invalidate because project 5c98a85ab7ea46189515430e8dc1b492 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:41.241010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-bd08db58-959b-4835-b620-b4df89ede70b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:41.241831 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-bd08db58-959b-4835-b620-b4df89ede70b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:41.243423 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1390/2761] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:40 2022] DELETE /identity/v3/projects/5c98a85ab7ea46189515430e8dc1b492 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:50.772155 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:DELETE=13,keystone:SELECT=65,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:51.235118 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=68,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:28.721698 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2d9b8-5589-4005-9798-14a90b2b83a7 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:28.723955 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-cad2d9b8-5589-4005-9798-14a90b2b83a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:28.842800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2d9b8-5589-4005-9798-14a90b2b83a7 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:28.921143 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6689aa4-6d0a-4ced-aabf-56bfc38b2dd9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07e3e25e6a8466fb237dcb836aecda2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:28.921790 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6689aa4-6d0a-4ced-aabf-56bfc38b2dd9 service placement] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:28.922144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6689aa4-6d0a-4ced-aabf-56bfc38b2dd9 service placement] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:28.922329 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6689aa4-6d0a-4ced-aabf-56bfc38b2dd9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:28.926173 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e6689aa4-6d0a-4ced-aabf-56bfc38b2dd9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:28.937937 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6689aa4-6d0a-4ced-aabf-56bfc38b2dd9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:28.939575 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6689aa4-6d0a-4ced-aabf-56bfc38b2dd9 service placement] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:28.952419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1372/2762] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:38.551912 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08db58-959b-4835-b620-b4df89ede70b admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:38.553592 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-bd08db58-959b-4835-b620-b4df89ede70b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:38.624452 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08db58-959b-4835-b620-b4df89ede70b admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:38.629411 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30254b78-66d3-459c-84da-d0ac0da44dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:38.630009 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30254b78-66d3-459c-84da-d0ac0da44dbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:38.630284 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30254b78-66d3-459c-84da-d0ac0da44dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:38.630558 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30254b78-66d3-459c-84da-d0ac0da44dbf admin admin] PATH_INFO: `/v3/users/9673eb00ccc24fbaa495ddcbaf6d3076` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:38.631884 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30254b78-66d3-459c-84da-d0ac0da44dbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9673eb00ccc24fbaa495ddcbaf6d3076)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:38.633880 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30254b78-66d3-459c-84da-d0ac0da44dbf admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:38.687087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-30254b78-66d3-459c-84da-d0ac0da44dbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9673eb00ccc24fbaa495ddcbaf6d3076'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:38.695678 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-30254b78-66d3-459c-84da-d0ac0da44dbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9673eb00ccc24fbaa495ddcbaf6d3076'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:38.705814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-30254b78-66d3-459c-84da-d0ac0da44dbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9673eb00ccc24fbaa495ddcbaf6d3076'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:38.706468 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-30254b78-66d3-459c-84da-d0ac0da44dbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9673eb00ccc24fbaa495ddcbaf6d3076'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:38.707272 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-30254b78-66d3-459c-84da-d0ac0da44dbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9673eb00ccc24fbaa495ddcbaf6d3076'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:38.728962 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1391/2763] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:25:38 2022] DELETE /identity/v3/users/9673eb00ccc24fbaa495ddcbaf6d3076 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:38.924549 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6689aa4-6d0a-4ced-aabf-56bfc38b2dd9 service placement] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:38.939310 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6689aa4-6d0a-4ced-aabf-56bfc38b2dd9 service placement] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:39.010994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ad843-5c6d-44d0-8ef8-9dbdd30f64f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:39.011744 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ad843-5c6d-44d0-8ef8-9dbdd30f64f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:39.011972 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ad843-5c6d-44d0-8ef8-9dbdd30f64f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:39.012250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ad843-5c6d-44d0-8ef8-9dbdd30f64f6 admin admin] PATH_INFO: `/v3/projects/f677d43502b044c981609fe0f0aed425` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:39.013554 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ad843-5c6d-44d0-8ef8-9dbdd30f64f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f677d43502b044c981609fe0f0aed425)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:39.015542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ad843-5c6d-44d0-8ef8-9dbdd30f64f6 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:39.059198 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:39.072921 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4d2ad843-5c6d-44d0-8ef8-9dbdd30f64f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f677d43502b044c981609fe0f0aed425', 'request_id': 'req-4d2ad843-5c6d-44d0-8ef8-9dbdd30f64f6'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:39.079927 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4d2ad843-5c6d-44d0-8ef8-9dbdd30f64f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f677d43502b044c981609fe0f0aed425', 'request_id': 'req-4d2ad843-5c6d-44d0-8ef8-9dbdd30f64f6'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:39.086890 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4d2ad843-5c6d-44d0-8ef8-9dbdd30f64f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f677d43502b044c981609fe0f0aed425', 'request_id': 'req-4d2ad843-5c6d-44d0-8ef8-9dbdd30f64f6'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:39.102018 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4d2ad843-5c6d-44d0-8ef8-9dbdd30f64f6 admin admin] The token cache is being invalidate because project f677d43502b044c981609fe0f0aed425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:25:39.102303 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4d2ad843-5c6d-44d0-8ef8-9dbdd30f64f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:39.102959 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-4d2ad843-5c6d-44d0-8ef8-9dbdd30f64f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:39.104582 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1373/2764] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:25:38 2022] DELETE /identity/v3/projects/f677d43502b044c981609fe0f0aed425 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:39.110855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fa578-8eb2-4c62-a935-86ac8ad8e429 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:39.111366 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fa578-8eb2-4c62-a935-86ac8ad8e429 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:39.111884 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fa578-8eb2-4c62-a935-86ac8ad8e429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:39.140933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-ae1fa578-8eb2-4c62-a935-86ac8ad8e429 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:39.142263 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ae1fa578-8eb2-4c62-a935-86ac8ad8e429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:39.210988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1392/2765] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:39.218984 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee26a0-f2d2-4ffa-b7fa-251283e4ed4a None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:39.219270 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee26a0-f2d2-4ffa-b7fa-251283e4ed4a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:39.219548 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee26a0-f2d2-4ffa-b7fa-251283e4ed4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:39.240256 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-a5ee26a0-f2d2-4ffa-b7fa-251283e4ed4a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:39.242006 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a5ee26a0-f2d2-4ffa-b7fa-251283e4ed4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:39.251269 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1374/2766] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:39.255789 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fa578-8eb2-4c62-a935-86ac8ad8e429 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:39.266857 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fa578-8eb2-4c62-a935-86ac8ad8e429 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:39.271786 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da9be7-d121-425f-a38f-c4c3b98e86ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:39.272419 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da9be7-d121-425f-a38f-c4c3b98e86ed admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:39.272739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da9be7-d121-425f-a38f-c4c3b98e86ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:39.273012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da9be7-d121-425f-a38f-c4c3b98e86ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:39.273750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da9be7-d121-425f-a38f-c4c3b98e86ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:25:39.274052 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da9be7-d121-425f-a38f-c4c3b98e86ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:39.275678 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da9be7-d121-425f-a38f-c4c3b98e86ed admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:39.289251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1393/2767] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:25:39.293844 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee26a0-f2d2-4ffa-b7fa-251283e4ed4a None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:39.304550 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee26a0-f2d2-4ffa-b7fa-251283e4ed4a None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:39.309981 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac680d3f-100f-421b-bad9-44aff775cd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:39.310614 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac680d3f-100f-421b-bad9-44aff775cd74 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:39.311040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac680d3f-100f-421b-bad9-44aff775cd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:39.311242 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac680d3f-100f-421b-bad9-44aff775cd74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:39.311809 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac680d3f-100f-421b-bad9-44aff775cd74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:39.314917 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac680d3f-100f-421b-bad9-44aff775cd74 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:39.369588 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1375/2768] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:25:39 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:25:39.374233 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da9be7-d121-425f-a38f-c4c3b98e86ed admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:39.389326 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da9be7-d121-425f-a38f-c4c3b98e86ed admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:39.396012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:39.468193 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cc96f-0da2-47e9-8cf0-304291e5a363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:39.468764 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cc96f-0da2-47e9-8cf0-304291e5a363 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:39.469041 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cc96f-0da2-47e9-8cf0-304291e5a363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:39.469312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cc96f-0da2-47e9-8cf0-304291e5a363 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:39.469987 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3cc96f-0da2-47e9-8cf0-304291e5a363 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:39.471919 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3cc96f-0da2-47e9-8cf0-304291e5a363 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:39.494979 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1394/2769] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:25:39 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:25:39.499457 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac680d3f-100f-421b-bad9-44aff775cd74 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:39.512822 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac680d3f-100f-421b-bad9-44aff775cd74 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:39.518281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ebf89-87a5-43f2-9b14-f5fbc2845e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:39.518910 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ebf89-87a5-43f2-9b14-f5fbc2845e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:39.519191 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ebf89-87a5-43f2-9b14-f5fbc2845e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:39.519460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ebf89-87a5-43f2-9b14-f5fbc2845e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:39.520155 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ebf89-87a5-43f2-9b14-f5fbc2845e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:39.521826 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ebf89-87a5-43f2-9b14-f5fbc2845e12 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:39.532883 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1376/2770] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:25:39.539599 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cc96f-0da2-47e9-8cf0-304291e5a363 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:39.558876 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cc96f-0da2-47e9-8cf0-304291e5a363 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:39.567070 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786641cc-a126-48a6-85b5-eebf5bf6a071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:39.568071 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786641cc-a126-48a6-85b5-eebf5bf6a071 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:39.568359 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786641cc-a126-48a6-85b5-eebf5bf6a071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:39.568874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786641cc-a126-48a6-85b5-eebf5bf6a071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:39.570180 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786641cc-a126-48a6-85b5-eebf5bf6a071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:39.573338 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786641cc-a126-48a6-85b5-eebf5bf6a071 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:39.594547 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1395/2771] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:25:39.598812 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ebf89-87a5-43f2-9b14-f5fbc2845e12 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:39.611201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ebf89-87a5-43f2-9b14-f5fbc2845e12 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:39.617672 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f3409-1e85-413a-854a-0bfdb2fb6bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:39.618609 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f3409-1e85-413a-854a-0bfdb2fb6bfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:39.618895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f3409-1e85-413a-854a-0bfdb2fb6bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:39.619289 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f3409-1e85-413a-854a-0bfdb2fb6bfd admin admin] PATH_INFO: `/v3/projects/f8c88d6d3b984e55a67453b186d219ec/users/f9a7719ad9f944438717aba8c74047ba/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:39.637907 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f3409-1e85-413a-854a-0bfdb2fb6bfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8c88d6d3b984e55a67453b186d219ec, user_id=f9a7719ad9f944438717aba8c74047ba, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:39.640662 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f3409-1e85-413a-854a-0bfdb2fb6bfd admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:39.657311 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1377/2772] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:39 2022] PUT /identity/v3/projects/f8c88d6d3b984e55a67453b186d219ec/users/f9a7719ad9f944438717aba8c74047ba/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:39.669645 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede52c5-d502-4471-8829-85fd0e8811a2 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:39.670291 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede52c5-d502-4471-8829-85fd0e8811a2 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:39.670772 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede52c5-d502-4471-8829-85fd0e8811a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:39.723321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-5ede52c5-d502-4471-8829-85fd0e8811a2 None None] MFA Rules not processed for user `f9a7719ad9f944438717aba8c74047ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:39.725468 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5ede52c5-d502-4471-8829-85fd0e8811a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:39.842058 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1396/2773] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:25:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:39.859352 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f3409-1e85-413a-854a-0bfdb2fb6bfd admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:39.861244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-e33f3409-1e85-413a-854a-0bfdb2fb6bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:39.953434 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f3409-1e85-413a-854a-0bfdb2fb6bfd admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:39.960230 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe52fc3-01c6-4d9f-bc16-24cdb2283897 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9be74621d14c94b95ae94947e5abbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:39.960898 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe52fc3-01c6-4d9f-bc16-24cdb2283897 service nova] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:39.961175 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe52fc3-01c6-4d9f-bc16-24cdb2283897 service nova] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:39.961441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe52fc3-01c6-4d9f-bc16-24cdb2283897 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:39.970520 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe52fc3-01c6-4d9f-bc16-24cdb2283897 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:39.972164 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe52fc3-01c6-4d9f-bc16-24cdb2283897 service nova] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:39.983638 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1378/2774] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:47.461696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede52c5-d502-4471-8829-85fd0e8811a2 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:47.549505 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede52c5-d502-4471-8829-85fd0e8811a2 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:47.557590 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a25ab-c037-461d-89be-e5c0db8946ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:47.558510 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a25ab-c037-461d-89be-e5c0db8946ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:47.558946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a25ab-c037-461d-89be-e5c0db8946ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:47.559378 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a25ab-c037-461d-89be-e5c0db8946ba admin admin] PATH_INFO: `/v3/users/f9a7719ad9f944438717aba8c74047ba` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:47.561442 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82a25ab-c037-461d-89be-e5c0db8946ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9a7719ad9f944438717aba8c74047ba)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:47.565036 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82a25ab-c037-461d-89be-e5c0db8946ba admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:47.593960 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:47.643502 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e82a25ab-c037-461d-89be-e5c0db8946ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9a7719ad9f944438717aba8c74047ba'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.653499 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e82a25ab-c037-461d-89be-e5c0db8946ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9a7719ad9f944438717aba8c74047ba'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.661196 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e82a25ab-c037-461d-89be-e5c0db8946ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9a7719ad9f944438717aba8c74047ba'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.661795 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e82a25ab-c037-461d-89be-e5c0db8946ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9a7719ad9f944438717aba8c74047ba'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.662395 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-e82a25ab-c037-461d-89be-e5c0db8946ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9a7719ad9f944438717aba8c74047ba'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.682730 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1397/2775] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:25:47 2022] DELETE /identity/v3/users/f9a7719ad9f944438717aba8c74047ba => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:47.687385 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48630fa4-da14-4fcf-983f-df93f69fa537 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:47.687963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48630fa4-da14-4fcf-983f-df93f69fa537 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:47.687963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48630fa4-da14-4fcf-983f-df93f69fa537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:47.709973 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-48630fa4-da14-4fcf-983f-df93f69fa537 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:47.711217 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-48630fa4-da14-4fcf-983f-df93f69fa537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:47.743864 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:47.783433 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1379/2776] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:25:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:47.795133 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a25ab-c037-461d-89be-e5c0db8946ba admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:47.797385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-e82a25ab-c037-461d-89be-e5c0db8946ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:47.867383 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a25ab-c037-461d-89be-e5c0db8946ba admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:47.871073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a37d9a-1326-440c-9ef3-c52071554490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:47.871733 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a37d9a-1326-440c-9ef3-c52071554490 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:47.872010 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a37d9a-1326-440c-9ef3-c52071554490 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:47.872283 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a37d9a-1326-440c-9ef3-c52071554490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:47.884724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a37d9a-1326-440c-9ef3-c52071554490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:47.886279 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a37d9a-1326-440c-9ef3-c52071554490 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:47.897512 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1398/2777] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:25:48.188245 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48630fa4-da14-4fcf-983f-df93f69fa537 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:48.189817 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-48630fa4-da14-4fcf-983f-df93f69fa537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:48.203365 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48630fa4-da14-4fcf-983f-df93f69fa537 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:48.208755 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a31e3d-1c70-4033-8fb4-17ee71fc3f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:48.209407 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a31e3d-1c70-4033-8fb4-17ee71fc3f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:48.209685 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a31e3d-1c70-4033-8fb4-17ee71fc3f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:48.209960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a31e3d-1c70-4033-8fb4-17ee71fc3f19 admin admin] PATH_INFO: `/v3/projects/f8c88d6d3b984e55a67453b186d219ec` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:48.211196 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a31e3d-1c70-4033-8fb4-17ee71fc3f19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8c88d6d3b984e55a67453b186d219ec)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:48.213942 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a31e3d-1c70-4033-8fb4-17ee71fc3f19 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:48.266143 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-54a31e3d-1c70-4033-8fb4-17ee71fc3f19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8c88d6d3b984e55a67453b186d219ec', 'request_id': 'req-54a31e3d-1c70-4033-8fb4-17ee71fc3f19'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.273058 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-54a31e3d-1c70-4033-8fb4-17ee71fc3f19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8c88d6d3b984e55a67453b186d219ec', 'request_id': 'req-54a31e3d-1c70-4033-8fb4-17ee71fc3f19'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.279369 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-54a31e3d-1c70-4033-8fb4-17ee71fc3f19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8c88d6d3b984e55a67453b186d219ec', 'request_id': 'req-54a31e3d-1c70-4033-8fb4-17ee71fc3f19'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.292586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-54a31e3d-1c70-4033-8fb4-17ee71fc3f19 admin admin] The token cache is being invalidate because project f8c88d6d3b984e55a67453b186d219ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:25:48.292877 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-54a31e3d-1c70-4033-8fb4-17ee71fc3f19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.293633 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-54a31e3d-1c70-4033-8fb4-17ee71fc3f19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.295707 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1380/2778] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:25:48 2022] DELETE /identity/v3/projects/f8c88d6d3b984e55a67453b186d219ec => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:48.301101 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc29aa0-0cf0-42c9-ba1f-fd72c8f4e61d None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:48.301393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc29aa0-0cf0-42c9-ba1f-fd72c8f4e61d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:48.301669 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc29aa0-0cf0-42c9-ba1f-fd72c8f4e61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:48.322227 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-3fc29aa0-0cf0-42c9-ba1f-fd72c8f4e61d None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:48.323399 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3fc29aa0-0cf0-42c9-ba1f-fd72c8f4e61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:48.399343 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1399/2779] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:25:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:48.407593 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bcf6d-b75d-48ff-b98a-fbe26774e50a None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:48.407870 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bcf6d-b75d-48ff-b98a-fbe26774e50a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:48.408161 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bcf6d-b75d-48ff-b98a-fbe26774e50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:48.428842 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-420bcf6d-b75d-48ff-b98a-fbe26774e50a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:48.430090 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-420bcf6d-b75d-48ff-b98a-fbe26774e50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:48.438230 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1381/2780] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:25:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:48.442732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc29aa0-0cf0-42c9-ba1f-fd72c8f4e61d None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:48.453761 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc29aa0-0cf0-42c9-ba1f-fd72c8f4e61d None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:48.458554 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56550f-3965-4bc0-9f73-2777c2f95560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:48.459169 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56550f-3965-4bc0-9f73-2777c2f95560 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:48.459495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56550f-3965-4bc0-9f73-2777c2f95560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:48.459863 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56550f-3965-4bc0-9f73-2777c2f95560 admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:48.460807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56550f-3965-4bc0-9f73-2777c2f95560 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:25:48.460807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56550f-3965-4bc0-9f73-2777c2f95560 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:48.462388 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56550f-3965-4bc0-9f73-2777c2f95560 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:48.474885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1400/2781] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:25:48.483960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420bcf6d-b75d-48ff-b98a-fbe26774e50a None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:48.494988 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420bcf6d-b75d-48ff-b98a-fbe26774e50a None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:48.499962 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92accbaa-64dd-44c4-b26a-93fc972dab6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:48.500629 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92accbaa-64dd-44c4-b26a-93fc972dab6e admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:48.500910 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92accbaa-64dd-44c4-b26a-93fc972dab6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:48.501209 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92accbaa-64dd-44c4-b26a-93fc972dab6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:48.501874 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92accbaa-64dd-44c4-b26a-93fc972dab6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:48.503802 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92accbaa-64dd-44c4-b26a-93fc972dab6e admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:48.535757 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1382/2782] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:25:48 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:48.540642 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56550f-3965-4bc0-9f73-2777c2f95560 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:48.550813 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56550f-3965-4bc0-9f73-2777c2f95560 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:48.572334 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:48.613325 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bed6c-f893-45c5-915f-0b8343296bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:48.614648 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bed6c-f893-45c5-915f-0b8343296bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:48.614648 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bed6c-f893-45c5-915f-0b8343296bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:48.614648 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bed6c-f893-45c5-915f-0b8343296bc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:48.615153 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4bed6c-f893-45c5-915f-0b8343296bc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:48.617714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4bed6c-f893-45c5-915f-0b8343296bc2 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:48.639207 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1401/2783] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:25:48 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:48.643807 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92accbaa-64dd-44c4-b26a-93fc972dab6e admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:48.660040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92accbaa-64dd-44c4-b26a-93fc972dab6e admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:48.668048 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30a454-d380-4c08-b8a2-23a96ed20d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:48.669162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30a454-d380-4c08-b8a2-23a96ed20d44 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:48.669667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30a454-d380-4c08-b8a2-23a96ed20d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:48.670159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30a454-d380-4c08-b8a2-23a96ed20d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:48.671334 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30a454-d380-4c08-b8a2-23a96ed20d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:48.673686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30a454-d380-4c08-b8a2-23a96ed20d44 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:48.685460 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1383/2784] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:25:48.690127 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bed6c-f893-45c5-915f-0b8343296bc2 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:48.701893 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bed6c-f893-45c5-915f-0b8343296bc2 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:48.707058 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080bb041-c33b-44e8-8111-ef84597e7c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:48.707711 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bb041-c33b-44e8-8111-ef84597e7c9a admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:48.708056 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bb041-c33b-44e8-8111-ef84597e7c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:48.708305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bb041-c33b-44e8-8111-ef84597e7c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:48.709160 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080bb041-c33b-44e8-8111-ef84597e7c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:48.711811 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080bb041-c33b-44e8-8111-ef84597e7c9a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:48.725333 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1402/2785] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:25:48.729725 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30a454-d380-4c08-b8a2-23a96ed20d44 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:48.739967 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30a454-d380-4c08-b8a2-23a96ed20d44 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:48.744823 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e90566-7719-479b-b442-4dba678c8368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:48.745474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e90566-7719-479b-b442-4dba678c8368 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:48.745747 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e90566-7719-479b-b442-4dba678c8368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:48.746017 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e90566-7719-479b-b442-4dba678c8368 admin admin] PATH_INFO: `/v3/projects/eca021f0a25849a4ab361354278427ce/users/8886dc0e530541a988bca4dc29be73ca/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:48.761184 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e90566-7719-479b-b442-4dba678c8368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eca021f0a25849a4ab361354278427ce, user_id=8886dc0e530541a988bca4dc29be73ca, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:48.763817 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e90566-7719-479b-b442-4dba678c8368 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:48.780763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1384/2786] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:48 2022] PUT /identity/v3/projects/eca021f0a25849a4ab361354278427ce/users/8886dc0e530541a988bca4dc29be73ca/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:48.791129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8c0b8-717c-490c-a3b4-e1203d9c6936 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:48.791417 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8c0b8-717c-490c-a3b4-e1203d9c6936 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:48.791720 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8c0b8-717c-490c-a3b4-e1203d9c6936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:48.820845 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-4bb8c0b8-717c-490c-a3b4-e1203d9c6936 None None] MFA Rules not processed for user `8886dc0e530541a988bca4dc29be73ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:48.822911 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4bb8c0b8-717c-490c-a3b4-e1203d9c6936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:48.894827 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1403/2787] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:25:48 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:48.906879 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e90566-7719-479b-b442-4dba678c8368 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:48.908787 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-78e90566-7719-479b-b442-4dba678c8368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:48.973719 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e90566-7719-479b-b442-4dba678c8368 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:48.977708 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a9eff-189e-4f36-802e-13c4cec1af4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:48.978321 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a9eff-189e-4f36-802e-13c4cec1af4f service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:48.978604 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a9eff-189e-4f36-802e-13c4cec1af4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:48.978881 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a9eff-189e-4f36-802e-13c4cec1af4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:48.987509 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a9eff-189e-4f36-802e-13c4cec1af4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:48.988987 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a9eff-189e-4f36-802e-13c4cec1af4f service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:48.993058 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:48.999379 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1385/2788] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:48 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:57.737432 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8c0b8-717c-490c-a3b4-e1203d9c6936 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:57.814232 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8c0b8-717c-490c-a3b4-e1203d9c6936 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:57.818878 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228419d-46c5-4bea-94bb-7c8e15deaf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:57.819525 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228419d-46c5-4bea-94bb-7c8e15deaf6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:57.819808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228419d-46c5-4bea-94bb-7c8e15deaf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:57.820102 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228419d-46c5-4bea-94bb-7c8e15deaf6a admin admin] PATH_INFO: `/v3/users/8886dc0e530541a988bca4dc29be73ca` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:57.821305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228419d-46c5-4bea-94bb-7c8e15deaf6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8886dc0e530541a988bca4dc29be73ca)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:57.823266 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228419d-46c5-4bea-94bb-7c8e15deaf6a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:57.869255 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:57.880933 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f228419d-46c5-4bea-94bb-7c8e15deaf6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8886dc0e530541a988bca4dc29be73ca'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:57.897709 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f228419d-46c5-4bea-94bb-7c8e15deaf6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8886dc0e530541a988bca4dc29be73ca'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:57.906329 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f228419d-46c5-4bea-94bb-7c8e15deaf6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8886dc0e530541a988bca4dc29be73ca'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:57.907159 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f228419d-46c5-4bea-94bb-7c8e15deaf6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8886dc0e530541a988bca4dc29be73ca'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:57.907870 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-f228419d-46c5-4bea-94bb-7c8e15deaf6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8886dc0e530541a988bca4dc29be73ca'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:57.928108 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1404/2789] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:25:57 2022] DELETE /identity/v3/users/8886dc0e530541a988bca4dc29be73ca => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:57.933308 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e5672-c653-4ae1-96ab-b14668242e1c None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:57.933863 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e5672-c653-4ae1-96ab-b14668242e1c None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:57.933863 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e5672-c653-4ae1-96ab-b14668242e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:57.957974 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-925e5672-c653-4ae1-96ab-b14668242e1c None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:57.959910 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-925e5672-c653-4ae1-96ab-b14668242e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:58.023857 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1386/2790] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:58.034258 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228419d-46c5-4bea-94bb-7c8e15deaf6a admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:58.035883 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-f228419d-46c5-4bea-94bb-7c8e15deaf6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:58.102988 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228419d-46c5-4bea-94bb-7c8e15deaf6a admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:58.106751 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636494e-4a5e-41c0-b84a-e68d99deaa2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:58.107374 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636494e-4a5e-41c0-b84a-e68d99deaa2c service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:58.107657 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636494e-4a5e-41c0-b84a-e68d99deaa2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:58.107946 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636494e-4a5e-41c0-b84a-e68d99deaa2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:58.118111 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636494e-4a5e-41c0-b84a-e68d99deaa2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:58.119562 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636494e-4a5e-41c0-b84a-e68d99deaa2c service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:58.130316 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1405/2791] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:58.433579 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e5672-c653-4ae1-96ab-b14668242e1c None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:58.435040 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-925e5672-c653-4ae1-96ab-b14668242e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:58.451267 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e5672-c653-4ae1-96ab-b14668242e1c None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:58.459722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd4bd4-2f52-47b6-a992-dda389777f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:58.460898 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd4bd4-2f52-47b6-a992-dda389777f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:58.461422 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd4bd4-2f52-47b6-a992-dda389777f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:58.461916 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd4bd4-2f52-47b6-a992-dda389777f14 admin admin] PATH_INFO: `/v3/projects/eca021f0a25849a4ab361354278427ce` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:58.464064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd4bd4-2f52-47b6-a992-dda389777f14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eca021f0a25849a4ab361354278427ce)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:58.467751 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd4bd4-2f52-47b6-a992-dda389777f14 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:58.527722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0bbd4bd4-2f52-47b6-a992-dda389777f14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eca021f0a25849a4ab361354278427ce', 'request_id': 'req-0bbd4bd4-2f52-47b6-a992-dda389777f14'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:58.535644 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0bbd4bd4-2f52-47b6-a992-dda389777f14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eca021f0a25849a4ab361354278427ce', 'request_id': 'req-0bbd4bd4-2f52-47b6-a992-dda389777f14'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:58.542301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0bbd4bd4-2f52-47b6-a992-dda389777f14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eca021f0a25849a4ab361354278427ce', 'request_id': 'req-0bbd4bd4-2f52-47b6-a992-dda389777f14'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:58.561880 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0bbd4bd4-2f52-47b6-a992-dda389777f14 admin admin] The token cache is being invalidate because project eca021f0a25849a4ab361354278427ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:25:58.562397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0bbd4bd4-2f52-47b6-a992-dda389777f14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:58.563348 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-0bbd4bd4-2f52-47b6-a992-dda389777f14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:58.565940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1387/2792] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:25:58 2022] DELETE /identity/v3/projects/eca021f0a25849a4ab361354278427ce => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:58.572590 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cf6ad-3bb7-4b8f-b532-d665044a4cac None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:58.572874 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cf6ad-3bb7-4b8f-b532-d665044a4cac None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:58.573148 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cf6ad-3bb7-4b8f-b532-d665044a4cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:58.599489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-fb7cf6ad-3bb7-4b8f-b532-d665044a4cac None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:58.601356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-fb7cf6ad-3bb7-4b8f-b532-d665044a4cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:58.673856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1406/2793] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:25:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:58.741731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f515c-5dfd-4ab2-8c03-a71d407edf36 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:58.742009 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f515c-5dfd-4ab2-8c03-a71d407edf36 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:58.742283 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f515c-5dfd-4ab2-8c03-a71d407edf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:58.762380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-a06f515c-5dfd-4ab2-8c03-a71d407edf36 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:58.763620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-a06f515c-5dfd-4ab2-8c03-a71d407edf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:58.772777 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1388/2794] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:25:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:58.777475 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7cf6ad-3bb7-4b8f-b532-d665044a4cac None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:58.787971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7cf6ad-3bb7-4b8f-b532-d665044a4cac None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:58.793133 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86a2d9-6e2d-4287-b4e0-b32873ed69ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:58.793739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86a2d9-6e2d-4287-b4e0-b32873ed69ce admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:58.794017 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86a2d9-6e2d-4287-b4e0-b32873ed69ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:58.794287 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86a2d9-6e2d-4287-b4e0-b32873ed69ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:58.794976 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86a2d9-6e2d-4287-b4e0-b32873ed69ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:25:58.795279 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86a2d9-6e2d-4287-b4e0-b32873ed69ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:58.796927 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86a2d9-6e2d-4287-b4e0-b32873ed69ce admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:58.809312 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1407/2795] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:25:58.813854 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f515c-5dfd-4ab2-8c03-a71d407edf36 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:58.824795 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f515c-5dfd-4ab2-8c03-a71d407edf36 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:58.829783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0eb78-1dd2-424e-8422-1d8caba7de40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:58.830408 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0eb78-1dd2-424e-8422-1d8caba7de40 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:58.830691 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0eb78-1dd2-424e-8422-1d8caba7de40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:58.830973 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0eb78-1dd2-424e-8422-1d8caba7de40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:58.831735 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0eb78-1dd2-424e-8422-1d8caba7de40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:58.833885 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0eb78-1dd2-424e-8422-1d8caba7de40 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:58.863909 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1389/2796] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:25:58 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:58.869732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86a2d9-6e2d-4287-b4e0-b32873ed69ce admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:58.879915 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86a2d9-6e2d-4287-b4e0-b32873ed69ce admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:58.921141 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:58.941660 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08ee5e-fd02-4144-83bf-e29e78480b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:58.942280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08ee5e-fd02-4144-83bf-e29e78480b85 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:58.942562 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08ee5e-fd02-4144-83bf-e29e78480b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:58.942835 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08ee5e-fd02-4144-83bf-e29e78480b85 admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:58.943499 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08ee5e-fd02-4144-83bf-e29e78480b85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:58.945461 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08ee5e-fd02-4144-83bf-e29e78480b85 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:58.967038 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1408/2797] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:25:58 2022] POST /identity/v3/users => generated 302 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:58.971381 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0eb78-1dd2-424e-8422-1d8caba7de40 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:58.982838 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0eb78-1dd2-424e-8422-1d8caba7de40 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:58.987487 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2ad33-5a59-4576-9dbd-561f8457f7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:58.988092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2ad33-5a59-4576-9dbd-561f8457f7d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:58.988486 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2ad33-5a59-4576-9dbd-561f8457f7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:58.988675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2ad33-5a59-4576-9dbd-561f8457f7d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:58.989374 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2ad33-5a59-4576-9dbd-561f8457f7d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:58.991173 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2ad33-5a59-4576-9dbd-561f8457f7d5 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:59.002766 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1390/2798] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:25:59.007385 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08ee5e-fd02-4144-83bf-e29e78480b85 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:59.018186 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08ee5e-fd02-4144-83bf-e29e78480b85 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:59.023749 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595cdda6-aa66-4adb-8e5a-4140b23a473c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:59.024356 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cdda6-aa66-4adb-8e5a-4140b23a473c admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:59.024662 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cdda6-aa66-4adb-8e5a-4140b23a473c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:59.024936 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cdda6-aa66-4adb-8e5a-4140b23a473c admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:59.025614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595cdda6-aa66-4adb-8e5a-4140b23a473c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:59.027362 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595cdda6-aa66-4adb-8e5a-4140b23a473c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:59.038413 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1409/2799] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:25:59.042686 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2ad33-5a59-4576-9dbd-561f8457f7d5 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:59.052760 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2ad33-5a59-4576-9dbd-561f8457f7d5 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:59.057353 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4e7b5-ac78-4e46-870a-6b8eecf4983e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:59.057999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4e7b5-ac78-4e46-870a-6b8eecf4983e admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:59.058274 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4e7b5-ac78-4e46-870a-6b8eecf4983e admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:59.058548 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4e7b5-ac78-4e46-870a-6b8eecf4983e admin admin] PATH_INFO: `/v3/projects/bb24600fa6374746bd428939e2c72526/users/39b5af2bafa742c2822a5c15a6a22869/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:59.074949 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4e7b5-ac78-4e46-870a-6b8eecf4983e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb24600fa6374746bd428939e2c72526, user_id=39b5af2bafa742c2822a5c15a6a22869, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:59.077647 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4e7b5-ac78-4e46-870a-6b8eecf4983e admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:59.091923 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1391/2800] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:59 2022] PUT /identity/v3/projects/bb24600fa6374746bd428939e2c72526/users/39b5af2bafa742c2822a5c15a6a22869/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:59.097084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4252606-9336-46e6-b6fc-7a9cc6541e76 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:59.097369 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4252606-9336-46e6-b6fc-7a9cc6541e76 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:59.097646 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4252606-9336-46e6-b6fc-7a9cc6541e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:59.117107 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-d4252606-9336-46e6-b6fc-7a9cc6541e76 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:59.118360 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-d4252606-9336-46e6-b6fc-7a9cc6541e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:59.178823 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1410/2801] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:59.188606 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4e7b5-ac78-4e46-870a-6b8eecf4983e admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:59.190354 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-bcb4e7b5-ac78-4e46-870a-6b8eecf4983e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:59.211345 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:59.254047 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4e7b5-ac78-4e46-870a-6b8eecf4983e admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:59.258469 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96cf8d-55e2-4127-bedf-6283fd9a0b16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:59.259084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96cf8d-55e2-4127-bedf-6283fd9a0b16 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:59.259365 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96cf8d-55e2-4127-bedf-6283fd9a0b16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:59.259638 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96cf8d-55e2-4127-bedf-6283fd9a0b16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:59.269000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96cf8d-55e2-4127-bedf-6283fd9a0b16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:59.270430 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96cf8d-55e2-4127-bedf-6283fd9a0b16 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:59.280337 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1392/2802] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:59.944099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c133269-36ac-40ba-b866-989ec7b5c193 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:59.944732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c133269-36ac-40ba-b866-989ec7b5c193 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:59.944732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c133269-36ac-40ba-b866-989ec7b5c193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:59.978606 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-7c133269-36ac-40ba-b866-989ec7b5c193 None None] MFA Rules not processed for user `39b5af2bafa742c2822a5c15a6a22869`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:59.979867 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-7c133269-36ac-40ba-b866-989ec7b5c193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:00.056043 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1411/2803] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:25:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:26:00.060857 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96cf8d-55e2-4127-bedf-6283fd9a0b16 service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:00.071168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96cf8d-55e2-4127-bedf-6283fd9a0b16 service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:00.075995 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e1220-0c7a-429a-a589-eafe591e8db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:00.076645 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e1220-0c7a-429a-a589-eafe591e8db9 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:00.076920 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e1220-0c7a-429a-a589-eafe591e8db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:00.077205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e1220-0c7a-429a-a589-eafe591e8db9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:00.077864 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e1220-0c7a-429a-a589-eafe591e8db9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:00.079765 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e1220-0c7a-429a-a589-eafe591e8db9 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:00.114294 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1393/2804] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:26:00 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:00.115919 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c133269-36ac-40ba-b866-989ec7b5c193 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:00.127155 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c133269-36ac-40ba-b866-989ec7b5c193 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:00.191399 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:00.207444 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ff3d6-f868-4fa9-9349-34a61b68ff6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:00.208075 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ff3d6-f868-4fa9-9349-34a61b68ff6e admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:00.208369 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ff3d6-f868-4fa9-9349-34a61b68ff6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:00.208661 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ff3d6-f868-4fa9-9349-34a61b68ff6e admin admin] PATH_INFO: `/v3/users` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:00.209995 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ff3d6-f868-4fa9-9349-34a61b68ff6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:00.211927 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ff3d6-f868-4fa9-9349-34a61b68ff6e admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:00.235683 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1412/2805] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:26:00 2022] POST /identity/v3/users => generated 308 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:00.242316 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e1220-0c7a-429a-a589-eafe591e8db9 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:00.261353 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e1220-0c7a-429a-a589-eafe591e8db9 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:00.269162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bf625-ddfa-4da0-9ce0-c5b36b69581b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:00.270244 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bf625-ddfa-4da0-9ce0-c5b36b69581b admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:00.270763 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bf625-ddfa-4da0-9ce0-c5b36b69581b admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:00.271288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bf625-ddfa-4da0-9ce0-c5b36b69581b admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:00.273147 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6bf625-ddfa-4da0-9ce0-c5b36b69581b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:00.276341 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6bf625-ddfa-4da0-9ce0-c5b36b69581b admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:00.297206 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1394/2806] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:00.303534 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ff3d6-f868-4fa9-9349-34a61b68ff6e admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:00.314862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ff3d6-f868-4fa9-9349-34a61b68ff6e admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:00.320571 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077cccd-cce9-4135-aeff-f6611d925b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:00.321252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077cccd-cce9-4135-aeff-f6611d925b7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:00.321533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077cccd-cce9-4135-aeff-f6611d925b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:00.321800 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077cccd-cce9-4135-aeff-f6611d925b7e admin admin] PATH_INFO: `/v3/projects/5a69cfa832ca430b910d1a631e76879a/users/63ece22cb95b44d6928a270501236428/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:00.340714 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077cccd-cce9-4135-aeff-f6611d925b7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a69cfa832ca430b910d1a631e76879a, user_id=63ece22cb95b44d6928a270501236428, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:00.343445 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077cccd-cce9-4135-aeff-f6611d925b7e admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:00.357607 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1413/2807] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:26:00 2022] PUT /identity/v3/projects/5a69cfa832ca430b910d1a631e76879a/users/63ece22cb95b44d6928a270501236428/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:00.361753 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bf625-ddfa-4da0-9ce0-c5b36b69581b admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:00.372198 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bf625-ddfa-4da0-9ce0-c5b36b69581b admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:00.432104 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a02ee8-4486-473f-8def-387ccdcb2ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:00.432780 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a02ee8-4486-473f-8def-387ccdcb2ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:00.433045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a02ee8-4486-473f-8def-387ccdcb2ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:00.433321 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a02ee8-4486-473f-8def-387ccdcb2ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:00.434023 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a02ee8-4486-473f-8def-387ccdcb2ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:00.435635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a02ee8-4486-473f-8def-387ccdcb2ae9 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:00.447432 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1395/2808] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:00.452026 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077cccd-cce9-4135-aeff-f6611d925b7e admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:00.463248 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077cccd-cce9-4135-aeff-f6611d925b7e admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:00.467917 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae19a8c-b3a1-42e6-ae2d-2d003c4af9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:00.468609 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae19a8c-b3a1-42e6-ae2d-2d003c4af9fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:00.468891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae19a8c-b3a1-42e6-ae2d-2d003c4af9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:00.469164 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae19a8c-b3a1-42e6-ae2d-2d003c4af9fa admin admin] PATH_INFO: `/v3/projects/5a69cfa832ca430b910d1a631e76879a/users/63ece22cb95b44d6928a270501236428/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:00.471110 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae19a8c-b3a1-42e6-ae2d-2d003c4af9fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a69cfa832ca430b910d1a631e76879a, user_id=63ece22cb95b44d6928a270501236428, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:00.473855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae19a8c-b3a1-42e6-ae2d-2d003c4af9fa admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:00.487548 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1414/2809] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:26:00 2022] PUT /identity/v3/projects/5a69cfa832ca430b910d1a631e76879a/users/63ece22cb95b44d6928a270501236428/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:01.181541 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab55bb-0c74-4f62-8560-f4312501e3cc None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:01.182168 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab55bb-0c74-4f62-8560-f4312501e3cc None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:01.182575 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab55bb-0c74-4f62-8560-f4312501e3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:01.217014 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-f4ab55bb-0c74-4f62-8560-f4312501e3cc None None] MFA Rules not processed for user `63ece22cb95b44d6928a270501236428`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:01.218401 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-f4ab55bb-0c74-4f62-8560-f4312501e3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:01.301048 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1396/2810] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:26:01 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:02.424882 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae19a8c-b3a1-42e6-ae2d-2d003c4af9fa admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:02.427631 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-eae19a8c-b3a1-42e6-ae2d-2d003c4af9fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:02.487382 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab55bb-0c74-4f62-8560-f4312501e3cc None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:02.501437 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:02.542097 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae19a8c-b3a1-42e6-ae2d-2d003c4af9fa admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:02.548811 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e932968-c58a-46e1-a51f-b153954fd4fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07e3e25e6a8466fb237dcb836aecda2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:02.549836 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e932968-c58a-46e1-a51f-b153954fd4fd service placement] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:02.550875 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e932968-c58a-46e1-a51f-b153954fd4fd service placement] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:02.550875 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e932968-c58a-46e1-a51f-b153954fd4fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:02.552707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3e932968-c58a-46e1-a51f-b153954fd4fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:02.571807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e932968-c58a-46e1-a51f-b153954fd4fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:02.573558 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e932968-c58a-46e1-a51f-b153954fd4fd service placement] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:02.585398 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1415/2811] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:02.586268 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab55bb-0c74-4f62-8560-f4312501e3cc None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:02.591964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b639da-b2de-42ad-b340-1a5888564060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:02.592706 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b639da-b2de-42ad-b340-1a5888564060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:02.592992 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b639da-b2de-42ad-b340-1a5888564060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:02.593269 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b639da-b2de-42ad-b340-1a5888564060 admin admin] PATH_INFO: `/v3/users/39b5af2bafa742c2822a5c15a6a22869` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:02.594538 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b639da-b2de-42ad-b340-1a5888564060 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39b5af2bafa742c2822a5c15a6a22869)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:02.596571 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b639da-b2de-42ad-b340-1a5888564060 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:02.653431 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-16b639da-b2de-42ad-b340-1a5888564060 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39b5af2bafa742c2822a5c15a6a22869'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:02.654260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-16b639da-b2de-42ad-b340-1a5888564060 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39b5af2bafa742c2822a5c15a6a22869'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:02.664166 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-16b639da-b2de-42ad-b340-1a5888564060 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39b5af2bafa742c2822a5c15a6a22869'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:02.664946 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-16b639da-b2de-42ad-b340-1a5888564060 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39b5af2bafa742c2822a5c15a6a22869'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:02.680158 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-16b639da-b2de-42ad-b340-1a5888564060 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39b5af2bafa742c2822a5c15a6a22869'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:02.694564 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1397/2812] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:26:02 2022] DELETE /identity/v3/users/39b5af2bafa742c2822a5c15a6a22869 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:02.894372 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e932968-c58a-46e1-a51f-b153954fd4fd service placement] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:02.906274 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e932968-c58a-46e1-a51f-b153954fd4fd service placement] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:02.968031 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25790c65-991e-4232-b2c2-959dbcad91f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:02.968823 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25790c65-991e-4232-b2c2-959dbcad91f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:02.969113 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25790c65-991e-4232-b2c2-959dbcad91f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:02.969387 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25790c65-991e-4232-b2c2-959dbcad91f8 admin admin] PATH_INFO: `/v3/projects/bb24600fa6374746bd428939e2c72526` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:02.970717 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25790c65-991e-4232-b2c2-959dbcad91f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb24600fa6374746bd428939e2c72526)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:02.972885 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25790c65-991e-4232-b2c2-959dbcad91f8 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:03.016507 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:03.029992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-25790c65-991e-4232-b2c2-959dbcad91f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb24600fa6374746bd428939e2c72526', 'request_id': 'req-25790c65-991e-4232-b2c2-959dbcad91f8'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:03.039064 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-25790c65-991e-4232-b2c2-959dbcad91f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb24600fa6374746bd428939e2c72526', 'request_id': 'req-25790c65-991e-4232-b2c2-959dbcad91f8'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:03.046697 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-25790c65-991e-4232-b2c2-959dbcad91f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb24600fa6374746bd428939e2c72526', 'request_id': 'req-25790c65-991e-4232-b2c2-959dbcad91f8'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:03.059760 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-25790c65-991e-4232-b2c2-959dbcad91f8 admin admin] The token cache is being invalidate because project bb24600fa6374746bd428939e2c72526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:26:03.060049 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-25790c65-991e-4232-b2c2-959dbcad91f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:03.060762 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-25790c65-991e-4232-b2c2-959dbcad91f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:03.062361 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1416/2813] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:26:02 2022] DELETE /identity/v3/projects/bb24600fa6374746bd428939e2c72526 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:03.067595 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b639da-b2de-42ad-b340-1a5888564060 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:03.069572 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-16b639da-b2de-42ad-b340-1a5888564060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:03.083940 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b639da-b2de-42ad-b340-1a5888564060 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:03.149466 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc078a2-9b0f-4fd7-a579-b611fa4710c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:03.150001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc078a2-9b0f-4fd7-a579-b611fa4710c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:03.150287 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc078a2-9b0f-4fd7-a579-b611fa4710c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:03.150558 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc078a2-9b0f-4fd7-a579-b611fa4710c4 admin admin] PATH_INFO: `/v3/users/63ece22cb95b44d6928a270501236428` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:03.152500 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc078a2-9b0f-4fd7-a579-b611fa4710c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63ece22cb95b44d6928a270501236428)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:03.154432 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc078a2-9b0f-4fd7-a579-b611fa4710c4 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:03.197835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:03.218326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6cc078a2-9b0f-4fd7-a579-b611fa4710c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63ece22cb95b44d6928a270501236428'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:03.218997 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6cc078a2-9b0f-4fd7-a579-b611fa4710c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63ece22cb95b44d6928a270501236428'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:03.227275 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6cc078a2-9b0f-4fd7-a579-b611fa4710c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63ece22cb95b44d6928a270501236428'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:03.227892 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6cc078a2-9b0f-4fd7-a579-b611fa4710c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63ece22cb95b44d6928a270501236428'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:03.240588 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-6cc078a2-9b0f-4fd7-a579-b611fa4710c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63ece22cb95b44d6928a270501236428'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:03.259625 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1398/2814] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:26:03 2022] DELETE /identity/v3/users/63ece22cb95b44d6928a270501236428 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:03.456742 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25790c65-991e-4232-b2c2-959dbcad91f8 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:03.458343 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-25790c65-991e-4232-b2c2-959dbcad91f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:03.473889 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25790c65-991e-4232-b2c2-959dbcad91f8 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:03.539693 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa813c-02bb-49ae-976b-a0d1959a067f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:03.540417 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa813c-02bb-49ae-976b-a0d1959a067f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:03.540825 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa813c-02bb-49ae-976b-a0d1959a067f admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:03.541498 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa813c-02bb-49ae-976b-a0d1959a067f admin admin] PATH_INFO: `/v3/projects/5a69cfa832ca430b910d1a631e76879a` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:03.542455 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa813c-02bb-49ae-976b-a0d1959a067f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a69cfa832ca430b910d1a631e76879a)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:03.544581 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa813c-02bb-49ae-976b-a0d1959a067f admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:03.594430 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d3aa813c-02bb-49ae-976b-a0d1959a067f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a69cfa832ca430b910d1a631e76879a', 'request_id': 'req-d3aa813c-02bb-49ae-976b-a0d1959a067f'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:03.603473 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d3aa813c-02bb-49ae-976b-a0d1959a067f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a69cfa832ca430b910d1a631e76879a', 'request_id': 'req-d3aa813c-02bb-49ae-976b-a0d1959a067f'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:03.609494 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d3aa813c-02bb-49ae-976b-a0d1959a067f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a69cfa832ca430b910d1a631e76879a', 'request_id': 'req-d3aa813c-02bb-49ae-976b-a0d1959a067f'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:03.622077 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d3aa813c-02bb-49ae-976b-a0d1959a067f admin admin] The token cache is being invalidate because project 5a69cfa832ca430b910d1a631e76879a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:26:03.622389 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d3aa813c-02bb-49ae-976b-a0d1959a067f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:03.623068 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-d3aa813c-02bb-49ae-976b-a0d1959a067f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:03.624658 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1417/2815] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:26:03 2022] DELETE /identity/v3/projects/5a69cfa832ca430b910d1a631e76879a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:03.629999 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cb1ca-d99f-4f2c-87e4-3bd3afc3ee32 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:03.630741 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cb1ca-d99f-4f2c-87e4-3bd3afc3ee32 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:03.631607 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cb1ca-d99f-4f2c-87e4-3bd3afc3ee32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:03.651522 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-0b3cb1ca-d99f-4f2c-87e4-3bd3afc3ee32 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:03.652780 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-0b3cb1ca-d99f-4f2c-87e4-3bd3afc3ee32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:03.720159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1399/2816] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:03.734139 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab02fbe-6831-4919-aa5b-682e9dfd04b5 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:03.734588 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab02fbe-6831-4919-aa5b-682e9dfd04b5 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:03.735029 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab02fbe-6831-4919-aa5b-682e9dfd04b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:03.761376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-5ab02fbe-6831-4919-aa5b-682e9dfd04b5 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:03.762628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-5ab02fbe-6831-4919-aa5b-682e9dfd04b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:03.772190 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1418/2817] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:03.777904 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cb1ca-d99f-4f2c-87e4-3bd3afc3ee32 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:03.788527 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cb1ca-d99f-4f2c-87e4-3bd3afc3ee32 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:03.793422 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42951097-4b2f-46f3-b3e0-a3620639b5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:03.794057 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42951097-4b2f-46f3-b3e0-a3620639b5be admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:03.794343 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42951097-4b2f-46f3-b3e0-a3620639b5be admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:03.794620 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42951097-4b2f-46f3-b3e0-a3620639b5be admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:03.795330 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42951097-4b2f-46f3-b3e0-a3620639b5be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:26:03.795636 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42951097-4b2f-46f3-b3e0-a3620639b5be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:03.797324 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42951097-4b2f-46f3-b3e0-a3620639b5be admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:03.811162 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1400/2818] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:26:03.815902 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab02fbe-6831-4919-aa5b-682e9dfd04b5 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:03.826822 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab02fbe-6831-4919-aa5b-682e9dfd04b5 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:03.832052 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8f140-e06a-4470-a622-242bb739bb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:03.832704 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8f140-e06a-4470-a622-242bb739bb5e admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:03.832987 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8f140-e06a-4470-a622-242bb739bb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:03.833258 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8f140-e06a-4470-a622-242bb739bb5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:03.833944 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8f140-e06a-4470-a622-242bb739bb5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:03.835878 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8f140-e06a-4470-a622-242bb739bb5e admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:03.866074 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1419/2819] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:26:03 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:03.870397 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42951097-4b2f-46f3-b3e0-a3620639b5be admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:03.883000 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42951097-4b2f-46f3-b3e0-a3620639b5be admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:03.965608 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f1396-ffc4-4613-8208-aab4372e92ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:03.966336 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f1396-ffc4-4613-8208-aab4372e92ab admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:03.966632 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f1396-ffc4-4613-8208-aab4372e92ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:03.966962 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f1396-ffc4-4613-8208-aab4372e92ab admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:03.967740 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f1396-ffc4-4613-8208-aab4372e92ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:03.969954 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f1396-ffc4-4613-8208-aab4372e92ab admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:03.998803 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1401/2820] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:26:03 2022] POST /identity/v3/users => generated 288 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:04.002767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8f140-e06a-4470-a622-242bb739bb5e admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:04.014621 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8f140-e06a-4470-a622-242bb739bb5e admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:04.022642 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6f20e-7610-4694-8256-6e50e1255e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:04.023273 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6f20e-7610-4694-8256-6e50e1255e42 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:04.023550 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6f20e-7610-4694-8256-6e50e1255e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:04.023838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6f20e-7610-4694-8256-6e50e1255e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:04.024570 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e6f20e-7610-4694-8256-6e50e1255e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:04.026205 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e6f20e-7610-4694-8256-6e50e1255e42 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:04.037136 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1420/2821] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:04.041794 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f1396-ffc4-4613-8208-aab4372e92ab admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:04.056667 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f1396-ffc4-4613-8208-aab4372e92ab admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:04.061712 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e15ee-962c-455c-973e-be4ff8fcd87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:04.062328 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e15ee-962c-455c-973e-be4ff8fcd87f admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:04.062605 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e15ee-962c-455c-973e-be4ff8fcd87f admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:04.062879 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e15ee-962c-455c-973e-be4ff8fcd87f admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:04.063576 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e15ee-962c-455c-973e-be4ff8fcd87f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:04.065220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e15ee-962c-455c-973e-be4ff8fcd87f admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:04.078386 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1402/2822] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:04.083168 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6f20e-7610-4694-8256-6e50e1255e42 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:04.093893 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6f20e-7610-4694-8256-6e50e1255e42 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:04.099079 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef43d5-9e6b-4b6e-bcc0-9e76f8e0e242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:04.100245 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef43d5-9e6b-4b6e-bcc0-9e76f8e0e242 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:04.100558 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef43d5-9e6b-4b6e-bcc0-9e76f8e0e242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:04.101101 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef43d5-9e6b-4b6e-bcc0-9e76f8e0e242 admin admin] PATH_INFO: `/v3/projects/0754ae04a6e24448a30ac3d31cc8c86c/users/a21b3a1822c742c68796a9c489d9d7e3/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:04.114412 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:04.123043 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef43d5-9e6b-4b6e-bcc0-9e76f8e0e242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0754ae04a6e24448a30ac3d31cc8c86c, user_id=a21b3a1822c742c68796a9c489d9d7e3, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:04.128939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef43d5-9e6b-4b6e-bcc0-9e76f8e0e242 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:04.145529 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1421/2823] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:26:04 2022] PUT /identity/v3/projects/0754ae04a6e24448a30ac3d31cc8c86c/users/a21b3a1822c742c68796a9c489d9d7e3/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:04.156423 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3d4d0-70d2-41ed-bc17-8a5309a6d06d None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:04.156738 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3d4d0-70d2-41ed-bc17-8a5309a6d06d None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:04.157012 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3d4d0-70d2-41ed-bc17-8a5309a6d06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:04.184402 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:04.188567 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-ebf3d4d0-70d2-41ed-bc17-8a5309a6d06d None None] MFA Rules not processed for user `a21b3a1822c742c68796a9c489d9d7e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:04.190580 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-ebf3d4d0-70d2-41ed-bc17-8a5309a6d06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:04.262805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1403/2824] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:26:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:04.269612 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef43d5-9e6b-4b6e-bcc0-9e76f8e0e242 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:04.341925 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef43d5-9e6b-4b6e-bcc0-9e76f8e0e242 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:04.346916 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77461d-a02b-4c5a-bd25-736c80778db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:04.347544 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77461d-a02b-4c5a-bd25-736c80778db9 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:04.347815 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77461d-a02b-4c5a-bd25-736c80778db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:04.348089 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77461d-a02b-4c5a-bd25-736c80778db9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:04.348808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77461d-a02b-4c5a-bd25-736c80778db9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:04.351724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77461d-a02b-4c5a-bd25-736c80778db9 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:04.381372 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1422/2825] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:26:04 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:04.385555 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3d4d0-70d2-41ed-bc17-8a5309a6d06d None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:04.397528 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3d4d0-70d2-41ed-bc17-8a5309a6d06d None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:04.467873 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d1d82-34e1-434f-85e7-f8712134f8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:04.468412 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d1d82-34e1-434f-85e7-f8712134f8ec admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:04.468736 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d1d82-34e1-434f-85e7-f8712134f8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:04.468983 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d1d82-34e1-434f-85e7-f8712134f8ec admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:04.469684 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d1d82-34e1-434f-85e7-f8712134f8ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:04.471899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d1d82-34e1-434f-85e7-f8712134f8ec admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:04.501467 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1404/2826] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:26:04 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:26:04.506575 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77461d-a02b-4c5a-bd25-736c80778db9 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:04.519339 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77461d-a02b-4c5a-bd25-736c80778db9 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:04.525453 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f9d6c-204f-4d95-a11a-d08dbf78f19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:04.526106 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f9d6c-204f-4d95-a11a-d08dbf78f19e admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:04.526387 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f9d6c-204f-4d95-a11a-d08dbf78f19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:04.526731 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f9d6c-204f-4d95-a11a-d08dbf78f19e admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:04.527390 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f9d6c-204f-4d95-a11a-d08dbf78f19e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:04.529108 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f9d6c-204f-4d95-a11a-d08dbf78f19e admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:04.545244 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1423/2827] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:04.551490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d1d82-34e1-434f-85e7-f8712134f8ec admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:04.565144 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d1d82-34e1-434f-85e7-f8712134f8ec admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:04.570933 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c773609-00af-499c-87e6-0b806c4b70df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:04.573096 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c773609-00af-499c-87e6-0b806c4b70df admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:04.573559 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c773609-00af-499c-87e6-0b806c4b70df admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:04.573844 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c773609-00af-499c-87e6-0b806c4b70df admin admin] PATH_INFO: `/v3/projects/a1a61c43356c4927a0e8c56f66015ad5/users/b5912d73eaeb4aca98e2e697f3a4bfa7/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:04.592854 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c773609-00af-499c-87e6-0b806c4b70df admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a61c43356c4927a0e8c56f66015ad5, user_id=b5912d73eaeb4aca98e2e697f3a4bfa7, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:04.595479 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c773609-00af-499c-87e6-0b806c4b70df admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:04.612114 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1405/2828] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:26:04 2022] PUT /identity/v3/projects/a1a61c43356c4927a0e8c56f66015ad5/users/b5912d73eaeb4aca98e2e697f3a4bfa7/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:26:04.616262 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f9d6c-204f-4d95-a11a-d08dbf78f19e admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:04.627352 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f9d6c-204f-4d95-a11a-d08dbf78f19e admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:04.687683 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dcb429-742d-4266-b915-ba6641e8131c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:04.688556 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcb429-742d-4266-b915-ba6641e8131c admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:04.688556 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcb429-742d-4266-b915-ba6641e8131c admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:04.688911 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcb429-742d-4266-b915-ba6641e8131c admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:04.689583 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dcb429-742d-4266-b915-ba6641e8131c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:04.691201 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dcb429-742d-4266-b915-ba6641e8131c admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:04.703328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1424/2829] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:04.708480 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c773609-00af-499c-87e6-0b806c4b70df admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:04.720601 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c773609-00af-499c-87e6-0b806c4b70df admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:04.726260 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da7ee7-2c21-49e7-ac9c-a52fb45f39b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:04.727064 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da7ee7-2c21-49e7-ac9c-a52fb45f39b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:04.727441 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da7ee7-2c21-49e7-ac9c-a52fb45f39b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:04.727836 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da7ee7-2c21-49e7-ac9c-a52fb45f39b6 admin admin] PATH_INFO: `/v3/projects/a1a61c43356c4927a0e8c56f66015ad5/users/b5912d73eaeb4aca98e2e697f3a4bfa7/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:04.730041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da7ee7-2c21-49e7-ac9c-a52fb45f39b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a61c43356c4927a0e8c56f66015ad5, user_id=b5912d73eaeb4aca98e2e697f3a4bfa7, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:04.732805 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da7ee7-2c21-49e7-ac9c-a52fb45f39b6 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:04.745084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1406/2830] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:26:04 2022] PUT /identity/v3/projects/a1a61c43356c4927a0e8c56f66015ad5/users/b5912d73eaeb4aca98e2e697f3a4bfa7/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:04.757763 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4e231-53a7-4ef5-8a6a-fb91170bc08c None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:04.758047 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4e231-53a7-4ef5-8a6a-fb91170bc08c None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:04.758321 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4e231-53a7-4ef5-8a6a-fb91170bc08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:04.788936 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-70f4e231-53a7-4ef5-8a6a-fb91170bc08c None None] MFA Rules not processed for user `b5912d73eaeb4aca98e2e697f3a4bfa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:04.790073 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-70f4e231-53a7-4ef5-8a6a-fb91170bc08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:04.840489 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:04.865130 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1425/2831] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:26:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:04.876993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da7ee7-2c21-49e7-ac9c-a52fb45f39b6 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:04.879047 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-f2da7ee7-2c21-49e7-ac9c-a52fb45f39b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:04.955382 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da7ee7-2c21-49e7-ac9c-a52fb45f39b6 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:04.959758 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d57dd-8b81-4c53-9651-7eb61042a745 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:04.960377 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d57dd-8b81-4c53-9651-7eb61042a745 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:04.960684 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d57dd-8b81-4c53-9651-7eb61042a745 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:04.960963 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d57dd-8b81-4c53-9651-7eb61042a745 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:04.969997 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:05.026516 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d57dd-8b81-4c53-9651-7eb61042a745 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:05.027970 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d57dd-8b81-4c53-9651-7eb61042a745 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:05.037479 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1407/2832] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:26:09.736777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4e231-53a7-4ef5-8a6a-fb91170bc08c None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:09.752368 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4e231-53a7-4ef5-8a6a-fb91170bc08c None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:09.756739 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be983da0-bea8-4816-9c78-3d469a802332 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:09.757433 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be983da0-bea8-4816-9c78-3d469a802332 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:09.757641 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be983da0-bea8-4816-9c78-3d469a802332 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:09.757923 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be983da0-bea8-4816-9c78-3d469a802332 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:09.768362 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be983da0-bea8-4816-9c78-3d469a802332 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:09.769891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be983da0-bea8-4816-9c78-3d469a802332 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:09.782191 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1426/2833] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:15.033038 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=23 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:19.776998 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=13 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:34.935635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d57dd-8b81-4c53-9651-7eb61042a745 service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:35.002535 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d57dd-8b81-4c53-9651-7eb61042a745 service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:35.008326 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd169a-4403-4e2f-bd9a-41ae9346fb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:35.009007 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd169a-4403-4e2f-bd9a-41ae9346fb07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:35.009287 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd169a-4403-4e2f-bd9a-41ae9346fb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:35.009558 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd169a-4403-4e2f-bd9a-41ae9346fb07 admin admin] PATH_INFO: `/v3/users/a21b3a1822c742c68796a9c489d9d7e3` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:35.010796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd169a-4403-4e2f-bd9a-41ae9346fb07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a21b3a1822c742c68796a9c489d9d7e3)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:35.012843 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd169a-4403-4e2f-bd9a-41ae9346fb07 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:35.065469 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-b6bd169a-4403-4e2f-bd9a-41ae9346fb07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a21b3a1822c742c68796a9c489d9d7e3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.066153 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-b6bd169a-4403-4e2f-bd9a-41ae9346fb07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a21b3a1822c742c68796a9c489d9d7e3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.074646 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-b6bd169a-4403-4e2f-bd9a-41ae9346fb07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a21b3a1822c742c68796a9c489d9d7e3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.075493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-b6bd169a-4403-4e2f-bd9a-41ae9346fb07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a21b3a1822c742c68796a9c489d9d7e3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.091598 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-b6bd169a-4403-4e2f-bd9a-41ae9346fb07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a21b3a1822c742c68796a9c489d9d7e3'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.106785 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1408/2834] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:26:34 2022] DELETE /identity/v3/users/a21b3a1822c742c68796a9c489d9d7e3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:35.112840 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a20fb0-b198-4d3a-8abf-34dcb7428100 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:35.113462 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a20fb0-b198-4d3a-8abf-34dcb7428100 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:35.113828 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a20fb0-b198-4d3a-8abf-34dcb7428100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:35.134668 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-75a20fb0-b198-4d3a-8abf-34dcb7428100 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:35.136088 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-75a20fb0-b198-4d3a-8abf-34dcb7428100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:35.199282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1427/2835] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:35.209635 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd169a-4403-4e2f-bd9a-41ae9346fb07 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:35.211159 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-b6bd169a-4403-4e2f-bd9a-41ae9346fb07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:35.271660 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd169a-4403-4e2f-bd9a-41ae9346fb07 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:35.275432 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a33ffd-8463-4930-817c-68199ed7f851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:35.276080 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a33ffd-8463-4930-817c-68199ed7f851 service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:35.276362 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a33ffd-8463-4930-817c-68199ed7f851 service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:35.276672 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a33ffd-8463-4930-817c-68199ed7f851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:35.287996 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a33ffd-8463-4930-817c-68199ed7f851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:35.289490 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a33ffd-8463-4930-817c-68199ed7f851 service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:35.299522 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1409/2836] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:26:35.597871 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a20fb0-b198-4d3a-8abf-34dcb7428100 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:35.599680 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-75a20fb0-b198-4d3a-8abf-34dcb7428100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:35.612585 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a20fb0-b198-4d3a-8abf-34dcb7428100 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:35.617389 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d8881-7c7d-4709-b6e3-b98fbd26ea44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:35.618034 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d8881-7c7d-4709-b6e3-b98fbd26ea44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:35.618308 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d8881-7c7d-4709-b6e3-b98fbd26ea44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:35.618576 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d8881-7c7d-4709-b6e3-b98fbd26ea44 admin admin] PATH_INFO: `/v3/projects/0754ae04a6e24448a30ac3d31cc8c86c` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:35.619956 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d8881-7c7d-4709-b6e3-b98fbd26ea44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0754ae04a6e24448a30ac3d31cc8c86c)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:35.621966 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d8881-7c7d-4709-b6e3-b98fbd26ea44 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:35.679951 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-bf9d8881-7c7d-4709-b6e3-b98fbd26ea44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0754ae04a6e24448a30ac3d31cc8c86c', 'request_id': 'req-bf9d8881-7c7d-4709-b6e3-b98fbd26ea44'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.688021 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-bf9d8881-7c7d-4709-b6e3-b98fbd26ea44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0754ae04a6e24448a30ac3d31cc8c86c', 'request_id': 'req-bf9d8881-7c7d-4709-b6e3-b98fbd26ea44'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.695093 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-bf9d8881-7c7d-4709-b6e3-b98fbd26ea44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0754ae04a6e24448a30ac3d31cc8c86c', 'request_id': 'req-bf9d8881-7c7d-4709-b6e3-b98fbd26ea44'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.706720 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-bf9d8881-7c7d-4709-b6e3-b98fbd26ea44 admin admin] The token cache is being invalidate because project 0754ae04a6e24448a30ac3d31cc8c86c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:26:35.706941 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-bf9d8881-7c7d-4709-b6e3-b98fbd26ea44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.707615 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-bf9d8881-7c7d-4709-b6e3-b98fbd26ea44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.709748 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1428/2837] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:26:35 2022] DELETE /identity/v3/projects/0754ae04a6e24448a30ac3d31cc8c86c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:35.714184 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a33ffd-8463-4930-817c-68199ed7f851 service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:35.724215 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a33ffd-8463-4930-817c-68199ed7f851 service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:35.743806 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:35.790259 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603089c8-0305-44ef-b8ff-2550afeba886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:35.790906 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603089c8-0305-44ef-b8ff-2550afeba886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:35.791190 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603089c8-0305-44ef-b8ff-2550afeba886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:35.791465 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603089c8-0305-44ef-b8ff-2550afeba886 admin admin] PATH_INFO: `/v3/users/b5912d73eaeb4aca98e2e697f3a4bfa7` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:35.792680 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603089c8-0305-44ef-b8ff-2550afeba886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5912d73eaeb4aca98e2e697f3a4bfa7)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:35.794607 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603089c8-0305-44ef-b8ff-2550afeba886 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:35.846847 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-603089c8-0305-44ef-b8ff-2550afeba886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5912d73eaeb4aca98e2e697f3a4bfa7'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.847543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-603089c8-0305-44ef-b8ff-2550afeba886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5912d73eaeb4aca98e2e697f3a4bfa7'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.856341 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-603089c8-0305-44ef-b8ff-2550afeba886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5912d73eaeb4aca98e2e697f3a4bfa7'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.857013 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-603089c8-0305-44ef-b8ff-2550afeba886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5912d73eaeb4aca98e2e697f3a4bfa7'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.871212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-603089c8-0305-44ef-b8ff-2550afeba886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5912d73eaeb4aca98e2e697f3a4bfa7'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.884672 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1410/2838] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:26:35 2022] DELETE /identity/v3/users/b5912d73eaeb4aca98e2e697f3a4bfa7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:36.169360 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d8881-7c7d-4709-b6e3-b98fbd26ea44 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:36.170906 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-bf9d8881-7c7d-4709-b6e3-b98fbd26ea44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:36.186840 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d8881-7c7d-4709-b6e3-b98fbd26ea44 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:36.260344 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fd957-3f3f-4b53-841d-e3d86a642698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:36.261208 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fd957-3f3f-4b53-841d-e3d86a642698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:36.261614 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fd957-3f3f-4b53-841d-e3d86a642698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:36.262118 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fd957-3f3f-4b53-841d-e3d86a642698 admin admin] PATH_INFO: `/v3/projects/a1a61c43356c4927a0e8c56f66015ad5` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:36.264016 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fd957-3f3f-4b53-841d-e3d86a642698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a61c43356c4927a0e8c56f66015ad5)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:36.266424 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fd957-3f3f-4b53-841d-e3d86a642698 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:36.305298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:36.326084 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-852fd957-3f3f-4b53-841d-e3d86a642698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a61c43356c4927a0e8c56f66015ad5', 'request_id': 'req-852fd957-3f3f-4b53-841d-e3d86a642698'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:36.334161 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-852fd957-3f3f-4b53-841d-e3d86a642698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a61c43356c4927a0e8c56f66015ad5', 'request_id': 'req-852fd957-3f3f-4b53-841d-e3d86a642698'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:36.340785 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-852fd957-3f3f-4b53-841d-e3d86a642698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a61c43356c4927a0e8c56f66015ad5', 'request_id': 'req-852fd957-3f3f-4b53-841d-e3d86a642698'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:36.353786 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-852fd957-3f3f-4b53-841d-e3d86a642698 admin admin] The token cache is being invalidate because project a1a61c43356c4927a0e8c56f66015ad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:26:36.354054 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-852fd957-3f3f-4b53-841d-e3d86a642698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:36.354732 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-852fd957-3f3f-4b53-841d-e3d86a642698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:36.358986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1429/2839] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:26:36 2022] DELETE /identity/v3/projects/a1a61c43356c4927a0e8c56f66015ad5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:36.365439 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed4290-2961-4ea2-a021-b783d0edec3e None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:36.366041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed4290-2961-4ea2-a021-b783d0edec3e None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:36.366041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed4290-2961-4ea2-a021-b783d0edec3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:36.387630 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-c5ed4290-2961-4ea2-a021-b783d0edec3e None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:36.388914 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-c5ed4290-2961-4ea2-a021-b783d0edec3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:36.449809 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1411/2840] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:36.458243 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e548863-5bb6-47d4-9435-14d512132ac2 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:36.458641 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e548863-5bb6-47d4-9435-14d512132ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:36.458809 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e548863-5bb6-47d4-9435-14d512132ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:36.480011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-0e548863-5bb6-47d4-9435-14d512132ac2 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:36.481267 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-0e548863-5bb6-47d4-9435-14d512132ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:36.489214 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1430/2841] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:36.494351 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed4290-2961-4ea2-a021-b783d0edec3e None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:36.504594 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed4290-2961-4ea2-a021-b783d0edec3e None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:36.511180 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db2a9d-cbaf-405f-b05d-d6c41440d284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:36.511765 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db2a9d-cbaf-405f-b05d-d6c41440d284 admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:36.512043 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db2a9d-cbaf-405f-b05d-d6c41440d284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:36.512317 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db2a9d-cbaf-405f-b05d-d6c41440d284 admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:36.513076 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db2a9d-cbaf-405f-b05d-d6c41440d284 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:26:36.513385 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db2a9d-cbaf-405f-b05d-d6c41440d284 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:36.515024 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db2a9d-cbaf-405f-b05d-d6c41440d284 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:36.527835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1412/2842] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:26:36.532448 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e548863-5bb6-47d4-9435-14d512132ac2 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:36.542635 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e548863-5bb6-47d4-9435-14d512132ac2 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:36.547297 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ceae05-1039-45ac-bb10-03d3cbbbe91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:36.547993 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ceae05-1039-45ac-bb10-03d3cbbbe91d admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:36.548262 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ceae05-1039-45ac-bb10-03d3cbbbe91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:36.548562 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ceae05-1039-45ac-bb10-03d3cbbbe91d admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:36.549219 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ceae05-1039-45ac-bb10-03d3cbbbe91d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:36.551142 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ceae05-1039-45ac-bb10-03d3cbbbe91d admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:36.580694 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1431/2843] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:26:36 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:36.585732 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db2a9d-cbaf-405f-b05d-d6c41440d284 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:36.598206 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db2a9d-cbaf-405f-b05d-d6c41440d284 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:36.628494 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:36.668957 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7ef71-267f-48d7-8001-d7ccc684abe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:36.669641 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7ef71-267f-48d7-8001-d7ccc684abe0 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:36.669960 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7ef71-267f-48d7-8001-d7ccc684abe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:36.670238 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7ef71-267f-48d7-8001-d7ccc684abe0 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:36.671069 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7ef71-267f-48d7-8001-d7ccc684abe0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:36.673405 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7ef71-267f-48d7-8001-d7ccc684abe0 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:36.696563 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1413/2844] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:26:36 2022] POST /identity/v3/users => generated 301 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:36.700860 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ceae05-1039-45ac-bb10-03d3cbbbe91d admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:36.713474 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ceae05-1039-45ac-bb10-03d3cbbbe91d admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:36.718404 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce40101-ee79-49d3-b0e5-4700cbc32404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:36.719017 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce40101-ee79-49d3-b0e5-4700cbc32404 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:36.719300 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce40101-ee79-49d3-b0e5-4700cbc32404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:36.719577 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce40101-ee79-49d3-b0e5-4700cbc32404 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:36.720267 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce40101-ee79-49d3-b0e5-4700cbc32404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:36.721912 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce40101-ee79-49d3-b0e5-4700cbc32404 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:36.733740 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1432/2845] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:36.737814 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7ef71-267f-48d7-8001-d7ccc684abe0 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:36.749541 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7ef71-267f-48d7-8001-d7ccc684abe0 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:36.754742 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd84140-7569-49a6-b3b3-d4e9923c8dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:36.755361 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd84140-7569-49a6-b3b3-d4e9923c8dcd admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:36.755678 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd84140-7569-49a6-b3b3-d4e9923c8dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:36.755910 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd84140-7569-49a6-b3b3-d4e9923c8dcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:36.756671 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd84140-7569-49a6-b3b3-d4e9923c8dcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:36.758321 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd84140-7569-49a6-b3b3-d4e9923c8dcd admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:36.769772 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1414/2846] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:36.774544 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce40101-ee79-49d3-b0e5-4700cbc32404 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:36.785129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce40101-ee79-49d3-b0e5-4700cbc32404 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:36.790476 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc035b57-fed6-45a6-b1a7-27359878c6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:36.791137 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc035b57-fed6-45a6-b1a7-27359878c6be admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:36.791419 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc035b57-fed6-45a6-b1a7-27359878c6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:36.791696 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc035b57-fed6-45a6-b1a7-27359878c6be admin admin] PATH_INFO: `/v3/projects/dd9518c9bff7436fb7c199f9efae9b3b/users/7a537de0ff53461780568ccaf9c7fec9/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:36.806769 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc035b57-fed6-45a6-b1a7-27359878c6be admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd9518c9bff7436fb7c199f9efae9b3b, user_id=7a537de0ff53461780568ccaf9c7fec9, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:36.809478 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc035b57-fed6-45a6-b1a7-27359878c6be admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:36.822923 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1433/2847] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:26:36 2022] PUT /identity/v3/projects/dd9518c9bff7436fb7c199f9efae9b3b/users/7a537de0ff53461780568ccaf9c7fec9/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:36.834758 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa5f3-ad20-4f67-a6db-a41eddb74265 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:36.835046 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa5f3-ad20-4f67-a6db-a41eddb74265 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:36.835325 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa5f3-ad20-4f67-a6db-a41eddb74265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:36.865714 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-054fa5f3-ad20-4f67-a6db-a41eddb74265 None None] MFA Rules not processed for user `7a537de0ff53461780568ccaf9c7fec9`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:36.866992 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-054fa5f3-ad20-4f67-a6db-a41eddb74265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:36.932909 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1415/2848] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:26:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:36.937501 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc035b57-fed6-45a6-b1a7-27359878c6be admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:37.000971 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc035b57-fed6-45a6-b1a7-27359878c6be admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:37.006632 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1a753-1d4e-4946-8255-d0995cab1914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:37.007256 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1a753-1d4e-4946-8255-d0995cab1914 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:37.007536 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1a753-1d4e-4946-8255-d0995cab1914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:37.007808 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1a753-1d4e-4946-8255-d0995cab1914 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:37.008482 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1a753-1d4e-4946-8255-d0995cab1914 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:37.010405 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1a753-1d4e-4946-8255-d0995cab1914 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:37.038549 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1434/2849] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:26:36 2022] POST /identity/v3/projects => generated 401 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:37.042665 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fa5f3-ad20-4f67-a6db-a41eddb74265 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:37.053559 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fa5f3-ad20-4f67-a6db-a41eddb74265 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:37.115186 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c4deb-c40b-4f36-88c7-f0351cf9751d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:37.115783 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c4deb-c40b-4f36-88c7-f0351cf9751d admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:37.116106 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c4deb-c40b-4f36-88c7-f0351cf9751d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:37.116329 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c4deb-c40b-4f36-88c7-f0351cf9751d admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:37.117084 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c4deb-c40b-4f36-88c7-f0351cf9751d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:37.119052 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c4deb-c40b-4f36-88c7-f0351cf9751d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:37.142083 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1416/2850] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:26:37 2022] POST /identity/v3/users => generated 307 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:26:37.147144 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1a753-1d4e-4946-8255-d0995cab1914 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:37.157831 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1a753-1d4e-4946-8255-d0995cab1914 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:37.163051 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d5ff8-a984-42a9-8c7f-523606b460b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:37.163673 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d5ff8-a984-42a9-8c7f-523606b460b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:37.163955 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d5ff8-a984-42a9-8c7f-523606b460b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:37.164229 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d5ff8-a984-42a9-8c7f-523606b460b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:37.164952 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d5ff8-a984-42a9-8c7f-523606b460b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:37.166551 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d5ff8-a984-42a9-8c7f-523606b460b9 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:37.178438 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1435/2851] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:37.182819 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c4deb-c40b-4f36-88c7-f0351cf9751d admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:37.195133 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c4deb-c40b-4f36-88c7-f0351cf9751d admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:37.200191 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357b5df-683a-4181-bda6-431941db8549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:37.200909 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357b5df-683a-4181-bda6-431941db8549 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:37.201184 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357b5df-683a-4181-bda6-431941db8549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:37.201454 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357b5df-683a-4181-bda6-431941db8549 admin admin] PATH_INFO: `/v3/projects/59a1fa6418a149c0b4c4f8d975a01f83/users/bdf244cd8de842b2adda66cc540e759e/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:37.217531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357b5df-683a-4181-bda6-431941db8549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a1fa6418a149c0b4c4f8d975a01f83, user_id=bdf244cd8de842b2adda66cc540e759e, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:37.220173 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357b5df-683a-4181-bda6-431941db8549 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:37.234330 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1417/2852] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:26:37 2022] PUT /identity/v3/projects/59a1fa6418a149c0b4c4f8d975a01f83/users/bdf244cd8de842b2adda66cc540e759e/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:37.239844 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d5ff8-a984-42a9-8c7f-523606b460b9 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:37.250635 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d5ff8-a984-42a9-8c7f-523606b460b9 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:37.272542 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:37.311855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6485a19-7119-4e8b-8941-1c20645e0d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:37.312376 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6485a19-7119-4e8b-8941-1c20645e0d2a admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:37.312710 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6485a19-7119-4e8b-8941-1c20645e0d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:37.312986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6485a19-7119-4e8b-8941-1c20645e0d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:37.313675 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6485a19-7119-4e8b-8941-1c20645e0d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:37.315265 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6485a19-7119-4e8b-8941-1c20645e0d2a admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:37.326082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1436/2853] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:26:37.330531 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357b5df-683a-4181-bda6-431941db8549 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:37.336799 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:37.349215 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357b5df-683a-4181-bda6-431941db8549 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:37.354708 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809aea76-b4e6-436f-a59a-69f8a9b03b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:37.355384 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809aea76-b4e6-436f-a59a-69f8a9b03b75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:37.355662 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809aea76-b4e6-436f-a59a-69f8a9b03b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:37.355941 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809aea76-b4e6-436f-a59a-69f8a9b03b75 admin admin] PATH_INFO: `/v3/projects/59a1fa6418a149c0b4c4f8d975a01f83/users/bdf244cd8de842b2adda66cc540e759e/roles/5456887a306649bf80a9ea7bd4282e97` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:37.358030 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809aea76-b4e6-436f-a59a-69f8a9b03b75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a1fa6418a149c0b4c4f8d975a01f83, user_id=bdf244cd8de842b2adda66cc540e759e, role_id=5456887a306649bf80a9ea7bd4282e97)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:37.360828 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809aea76-b4e6-436f-a59a-69f8a9b03b75 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:37.372738 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1418/2854] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:26:37 2022] PUT /identity/v3/projects/59a1fa6418a149c0b4c4f8d975a01f83/users/bdf244cd8de842b2adda66cc540e759e/roles/5456887a306649bf80a9ea7bd4282e97 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:37.384101 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112744c-6871-441f-8d47-12e0f2c74f5b None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:37.384382 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112744c-6871-441f-8d47-12e0f2c74f5b None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:37.384689 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112744c-6871-441f-8d47-12e0f2c74f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:37.414408 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-4112744c-6871-441f-8d47-12e0f2c74f5b None None] MFA Rules not processed for user `bdf244cd8de842b2adda66cc540e759e`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:37.415649 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-4112744c-6871-441f-8d47-12e0f2c74f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:37.499986 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1437/2855] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:26:37 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:37.514991 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809aea76-b4e6-436f-a59a-69f8a9b03b75 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:37.517177 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-809aea76-b4e6-436f-a59a-69f8a9b03b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:37.589516 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809aea76-b4e6-436f-a59a-69f8a9b03b75 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:37.594018 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b8d00-a944-4643-b74d-ff99f2961789 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:37.594685 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b8d00-a944-4643-b74d-ff99f2961789 service cinder] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:37.594961 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b8d00-a944-4643-b74d-ff99f2961789 service cinder] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:37.595237 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b8d00-a944-4643-b74d-ff99f2961789 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:37.607093 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b8d00-a944-4643-b74d-ff99f2961789 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:37.608578 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b8d00-a944-4643-b74d-ff99f2961789 service cinder] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:37.619197 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1419/2856] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:37 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:37.699502 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112744c-6871-441f-8d47-12e0f2c74f5b None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:37.710055 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112744c-6871-441f-8d47-12e0f2c74f5b None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:37.713716 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cfe90-5fb5-4846-8850-ea862ce8a3d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:37.714341 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cfe90-5fb5-4846-8850-ea862ce8a3d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:37.714621 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cfe90-5fb5-4846-8850-ea862ce8a3d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:37.714895 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cfe90-5fb5-4846-8850-ea862ce8a3d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:37.775832 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6cfe90-5fb5-4846-8850-ea862ce8a3d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:37.777305 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6cfe90-5fb5-4846-8850-ea862ce8a3d4 service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:37.787687 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1438/2857] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:41.937961 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b8d00-a944-4643-b74d-ff99f2961789 service cinder] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:42.005802 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b8d00-a944-4643-b74d-ff99f2961789 service cinder] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:42.010586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f91b4f-8c0f-44e4-a100-3ac89920ab90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:42.011232 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f91b4f-8c0f-44e4-a100-3ac89920ab90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:42.011511 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f91b4f-8c0f-44e4-a100-3ac89920ab90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:42.011787 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f91b4f-8c0f-44e4-a100-3ac89920ab90 admin admin] PATH_INFO: `/v3/users/7a537de0ff53461780568ccaf9c7fec9` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:42.013004 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f91b4f-8c0f-44e4-a100-3ac89920ab90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a537de0ff53461780568ccaf9c7fec9)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:42.014930 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f91b4f-8c0f-44e4-a100-3ac89920ab90 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:42.065900 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-02f91b4f-8c0f-44e4-a100-3ac89920ab90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a537de0ff53461780568ccaf9c7fec9'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:42.066568 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-02f91b4f-8c0f-44e4-a100-3ac89920ab90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a537de0ff53461780568ccaf9c7fec9'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:42.075847 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-02f91b4f-8c0f-44e4-a100-3ac89920ab90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a537de0ff53461780568ccaf9c7fec9'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:42.076484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-02f91b4f-8c0f-44e4-a100-3ac89920ab90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a537de0ff53461780568ccaf9c7fec9'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:42.089493 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-02f91b4f-8c0f-44e4-a100-3ac89920ab90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a537de0ff53461780568ccaf9c7fec9'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:42.112964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1420/2858] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:26:41 2022] DELETE /identity/v3/users/7a537de0ff53461780568ccaf9c7fec9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:42.119363 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446feff-a370-4485-aca2-7594889067ad None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:42.119912 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446feff-a370-4485-aca2-7594889067ad None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:42.120016 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446feff-a370-4485-aca2-7594889067ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:42.142176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-a446feff-a370-4485-aca2-7594889067ad None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:42.143451 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a446feff-a370-4485-aca2-7594889067ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:42.191629 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=100 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:42.218997 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1439/2859] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:42.230204 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f91b4f-8c0f-44e4-a100-3ac89920ab90 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:42.231665 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-02f91b4f-8c0f-44e4-a100-3ac89920ab90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:42.246640 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:42.299370 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f91b4f-8c0f-44e4-a100-3ac89920ab90 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:42.303810 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42178025-e2d3-4465-8e49-d42c23a6ea9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:42.304447 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42178025-e2d3-4465-8e49-d42c23a6ea9a service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:42.304770 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42178025-e2d3-4465-8e49-d42c23a6ea9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:42.305047 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42178025-e2d3-4465-8e49-d42c23a6ea9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:42.315481 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42178025-e2d3-4465-8e49-d42c23a6ea9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:42.317481 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42178025-e2d3-4465-8e49-d42c23a6ea9a service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:42.336395 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1421/2860] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:42.626154 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446feff-a370-4485-aca2-7594889067ad None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:42.627898 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-a446feff-a370-4485-aca2-7594889067ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:42.644987 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446feff-a370-4485-aca2-7594889067ad None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:42.653752 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ba394-fe68-4029-a4b8-863fc6805277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:42.654939 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ba394-fe68-4029-a4b8-863fc6805277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:42.655350 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ba394-fe68-4029-a4b8-863fc6805277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:42.655724 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ba394-fe68-4029-a4b8-863fc6805277 admin admin] PATH_INFO: `/v3/projects/dd9518c9bff7436fb7c199f9efae9b3b` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:42.657513 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ba394-fe68-4029-a4b8-863fc6805277 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd9518c9bff7436fb7c199f9efae9b3b)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:42.660098 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ba394-fe68-4029-a4b8-863fc6805277 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:42.708667 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-228ba394-fe68-4029-a4b8-863fc6805277 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd9518c9bff7436fb7c199f9efae9b3b', 'request_id': 'req-228ba394-fe68-4029-a4b8-863fc6805277'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:42.716192 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-228ba394-fe68-4029-a4b8-863fc6805277 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd9518c9bff7436fb7c199f9efae9b3b', 'request_id': 'req-228ba394-fe68-4029-a4b8-863fc6805277'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:42.722001 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-228ba394-fe68-4029-a4b8-863fc6805277 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd9518c9bff7436fb7c199f9efae9b3b', 'request_id': 'req-228ba394-fe68-4029-a4b8-863fc6805277'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:42.734761 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-228ba394-fe68-4029-a4b8-863fc6805277 admin admin] The token cache is being invalidate because project dd9518c9bff7436fb7c199f9efae9b3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:26:42.735057 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-228ba394-fe68-4029-a4b8-863fc6805277 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:42.735707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-228ba394-fe68-4029-a4b8-863fc6805277 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:42.737302 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1440/2861] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:26:42 2022] DELETE /identity/v3/projects/dd9518c9bff7436fb7c199f9efae9b3b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:42.741344 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42178025-e2d3-4465-8e49-d42c23a6ea9a service neutron] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:42.750994 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42178025-e2d3-4465-8e49-d42c23a6ea9a service neutron] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:42.808568 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb638d-5c3a-4a5b-93cf-efc176fabac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:42.809205 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb638d-5c3a-4a5b-93cf-efc176fabac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:42.809484 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb638d-5c3a-4a5b-93cf-efc176fabac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:42.809755 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb638d-5c3a-4a5b-93cf-efc176fabac9 admin admin] PATH_INFO: `/v3/users/bdf244cd8de842b2adda66cc540e759e` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:42.810905 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb638d-5c3a-4a5b-93cf-efc176fabac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdf244cd8de842b2adda66cc540e759e)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:42.812868 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb638d-5c3a-4a5b-93cf-efc176fabac9 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:42.861755 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c4eb638d-5c3a-4a5b-93cf-efc176fabac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdf244cd8de842b2adda66cc540e759e'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:42.862800 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c4eb638d-5c3a-4a5b-93cf-efc176fabac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdf244cd8de842b2adda66cc540e759e'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:42.873280 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c4eb638d-5c3a-4a5b-93cf-efc176fabac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdf244cd8de842b2adda66cc540e759e'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:42.873897 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c4eb638d-5c3a-4a5b-93cf-efc176fabac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdf244cd8de842b2adda66cc540e759e'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:42.887599 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-c4eb638d-5c3a-4a5b-93cf-efc176fabac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdf244cd8de842b2adda66cc540e759e'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:42.902230 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1422/2862] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:26:42 2022] DELETE /identity/v3/users/bdf244cd8de842b2adda66cc540e759e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:43.185907 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ba394-fe68-4029-a4b8-863fc6805277 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:43.187633 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-228ba394-fe68-4029-a4b8-863fc6805277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:43.202135 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ba394-fe68-4029-a4b8-863fc6805277 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:43.263475 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab58072-2505-4818-930f-dbaae1374f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:43.264155 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab58072-2505-4818-930f-dbaae1374f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:43.264393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab58072-2505-4818-930f-dbaae1374f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:43.264695 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab58072-2505-4818-930f-dbaae1374f5d admin admin] PATH_INFO: `/v3/projects/59a1fa6418a149c0b4c4f8d975a01f83` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:43.265881 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab58072-2505-4818-930f-dbaae1374f5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=59a1fa6418a149c0b4c4f8d975a01f83)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:43.267784 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab58072-2505-4818-930f-dbaae1374f5d admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:43.322414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-dab58072-2505-4818-930f-dbaae1374f5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59a1fa6418a149c0b4c4f8d975a01f83', 'request_id': 'req-dab58072-2505-4818-930f-dbaae1374f5d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:43.330230 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-dab58072-2505-4818-930f-dbaae1374f5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59a1fa6418a149c0b4c4f8d975a01f83', 'request_id': 'req-dab58072-2505-4818-930f-dbaae1374f5d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:43.338087 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-dab58072-2505-4818-930f-dbaae1374f5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59a1fa6418a149c0b4c4f8d975a01f83', 'request_id': 'req-dab58072-2505-4818-930f-dbaae1374f5d'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:43.346122 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:43.352619 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-dab58072-2505-4818-930f-dbaae1374f5d admin admin] The token cache is being invalidate because project 59a1fa6418a149c0b4c4f8d975a01f83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:26:43.353268 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-dab58072-2505-4818-930f-dbaae1374f5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:43.353992 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-dab58072-2505-4818-930f-dbaae1374f5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:43.356485 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1441/2863] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:26:43 2022] DELETE /identity/v3/projects/59a1fa6418a149c0b4c4f8d975a01f83 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:43.363474 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e6b5d-68c1-4c99-9edc-bd1c15d5904a None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:43.365208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e6b5d-68c1-4c99-9edc-bd1c15d5904a None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:43.365208 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e6b5d-68c1-4c99-9edc-bd1c15d5904a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:43.387188 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-838e6b5d-68c1-4c99-9edc-bd1c15d5904a None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:43.388212 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-838e6b5d-68c1-4c99-9edc-bd1c15d5904a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:43.421700 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:43.481098 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1423/2864] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:43.490041 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4ac84-d259-413e-a4a0-714c502a8e8b None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:43.490345 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4ac84-d259-413e-a4a0-714c502a8e8b None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:43.490628 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4ac84-d259-413e-a4a0-714c502a8e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:43.515373 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-2de4ac84-d259-413e-a4a0-714c502a8e8b None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:43.516699 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-2de4ac84-d259-413e-a4a0-714c502a8e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:43.525110 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1442/2865] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:43.529865 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e6b5d-68c1-4c99-9edc-bd1c15d5904a None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:43.540092 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e6b5d-68c1-4c99-9edc-bd1c15d5904a None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:43.544922 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0708ee-4288-4a45-9cac-44e441348b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:43.545536 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0708ee-4288-4a45-9cac-44e441348b1d admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:43.545812 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0708ee-4288-4a45-9cac-44e441348b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:43.546082 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0708ee-4288-4a45-9cac-44e441348b1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:43.546785 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0708ee-4288-4a45-9cac-44e441348b1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:26:43.547112 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0708ee-4288-4a45-9cac-44e441348b1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:43.548749 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0708ee-4288-4a45-9cac-44e441348b1d admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:43.562039 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1424/2866] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:26:43.566556 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4ac84-d259-413e-a4a0-714c502a8e8b None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:43.576859 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4ac84-d259-413e-a4a0-714c502a8e8b None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:43.581598 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7edf5a-e91a-4f24-825d-4a00137f5c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:43.582212 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7edf5a-e91a-4f24-825d-4a00137f5c71 admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:43.582487 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7edf5a-e91a-4f24-825d-4a00137f5c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:43.582759 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7edf5a-e91a-4f24-825d-4a00137f5c71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:43.583427 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7edf5a-e91a-4f24-825d-4a00137f5c71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:43.585355 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7edf5a-e91a-4f24-825d-4a00137f5c71 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:43.615428 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1443/2867] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:26:43 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:26:43.619143 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0708ee-4288-4a45-9cac-44e441348b1d admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:43.629500 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0708ee-4288-4a45-9cac-44e441348b1d admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:43.689267 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da911285-22e1-471d-ac9c-4f27d07b8b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:43.689796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da911285-22e1-471d-ac9c-4f27d07b8b30 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:43.690080 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da911285-22e1-471d-ac9c-4f27d07b8b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:43.690359 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da911285-22e1-471d-ac9c-4f27d07b8b30 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:43.691025 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da911285-22e1-471d-ac9c-4f27d07b8b30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:43.693697 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da911285-22e1-471d-ac9c-4f27d07b8b30 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:43.715848 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1425/2868] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:26:43 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:43.720171 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7edf5a-e91a-4f24-825d-4a00137f5c71 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:43.731506 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7edf5a-e91a-4f24-825d-4a00137f5c71 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:43.736353 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bcfc04-f225-4882-ab06-95d62d7ad609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:43.737006 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcfc04-f225-4882-ab06-95d62d7ad609 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:43.737293 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcfc04-f225-4882-ab06-95d62d7ad609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:43.737565 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcfc04-f225-4882-ab06-95d62d7ad609 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:43.738253 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bcfc04-f225-4882-ab06-95d62d7ad609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:43.740441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bcfc04-f225-4882-ab06-95d62d7ad609 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:43.751027 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1444/2869] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:43.756166 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da911285-22e1-471d-ac9c-4f27d07b8b30 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:43.767668 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da911285-22e1-471d-ac9c-4f27d07b8b30 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:43.775201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0af110-580f-4c77-8787-b93e7e3edeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:43.776132 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0af110-580f-4c77-8787-b93e7e3edeec admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:43.776415 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0af110-580f-4c77-8787-b93e7e3edeec admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:43.776727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0af110-580f-4c77-8787-b93e7e3edeec admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:43.777431 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0af110-580f-4c77-8787-b93e7e3edeec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:43.779045 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0af110-580f-4c77-8787-b93e7e3edeec admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:43.791794 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1426/2870] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:43.796337 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bcfc04-f225-4882-ab06-95d62d7ad609 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:43.807033 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bcfc04-f225-4882-ab06-95d62d7ad609 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:43.811838 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d6ed5-732f-47c2-b396-55b9bb586c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:43.812532 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d6ed5-732f-47c2-b396-55b9bb586c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:43.812814 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d6ed5-732f-47c2-b396-55b9bb586c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:43.813092 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d6ed5-732f-47c2-b396-55b9bb586c70 admin admin] PATH_INFO: `/v3/projects/758e37f4382744d1b4546345e431ee0a/users/fa7d630b64264524a345666f7d434ee6/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:43.829176 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d6ed5-732f-47c2-b396-55b9bb586c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=758e37f4382744d1b4546345e431ee0a, user_id=fa7d630b64264524a345666f7d434ee6, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:43.831859 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d6ed5-732f-47c2-b396-55b9bb586c70 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:43.845306 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1445/2871] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:26:43 2022] PUT /identity/v3/projects/758e37f4382744d1b4546345e431ee0a/users/fa7d630b64264524a345666f7d434ee6/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:43.858033 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aa8b4-05a2-46b7-89b4-90f9ae776060 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:43.858522 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aa8b4-05a2-46b7-89b4-90f9ae776060 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:43.858899 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aa8b4-05a2-46b7-89b4-90f9ae776060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:43.888759 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-482aa8b4-05a2-46b7-89b4-90f9ae776060 None None] MFA Rules not processed for user `fa7d630b64264524a345666f7d434ee6`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:43.890001 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-482aa8b4-05a2-46b7-89b4-90f9ae776060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:43.954068 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1427/2872] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:26:43 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:26:43.965677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d6ed5-732f-47c2-b396-55b9bb586c70 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:43.967518 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-712d6ed5-732f-47c2-b396-55b9bb586c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:44.047464 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d6ed5-732f-47c2-b396-55b9bb586c70 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:44.051278 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67042bb5-e964-482a-a490-accb7763c8b2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4461a389da5848f5afd9854758707829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:44.052450 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67042bb5-e964-482a-a490-accb7763c8b2 service glance] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:44.052767 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67042bb5-e964-482a-a490-accb7763c8b2 service glance] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:44.053040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67042bb5-e964-482a-a490-accb7763c8b2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:44.062107 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67042bb5-e964-482a-a490-accb7763c8b2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:44.063530 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67042bb5-e964-482a-a490-accb7763c8b2 service glance] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:44.073046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1446/2873] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:43 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:53.951041 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:54.068935 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=63,keystone:INSERT=2 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:16.610419 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482aa8b4-05a2-46b7-89b4-90f9ae776060 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:16.673476 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482aa8b4-05a2-46b7-89b4-90f9ae776060 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:16.677967 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62505e11-a26e-4300-b86a-6f90176a5e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:16.678598 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62505e11-a26e-4300-b86a-6f90176a5e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:16.678869 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62505e11-a26e-4300-b86a-6f90176a5e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:16.679137 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62505e11-a26e-4300-b86a-6f90176a5e84 admin admin] PATH_INFO: `/v3/users/fa7d630b64264524a345666f7d434ee6` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:16.680290 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62505e11-a26e-4300-b86a-6f90176a5e84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa7d630b64264524a345666f7d434ee6)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:16.682271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62505e11-a26e-4300-b86a-6f90176a5e84 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:16.731088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-62505e11-a26e-4300-b86a-6f90176a5e84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa7d630b64264524a345666f7d434ee6'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:16.731748 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-62505e11-a26e-4300-b86a-6f90176a5e84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa7d630b64264524a345666f7d434ee6'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:16.740210 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-62505e11-a26e-4300-b86a-6f90176a5e84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa7d630b64264524a345666f7d434ee6'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:16.741077 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-62505e11-a26e-4300-b86a-6f90176a5e84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa7d630b64264524a345666f7d434ee6'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:16.755501 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-62505e11-a26e-4300-b86a-6f90176a5e84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa7d630b64264524a345666f7d434ee6'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:16.773388 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1428/2874] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:27:16 2022] DELETE /identity/v3/users/fa7d630b64264524a345666f7d434ee6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:16.777893 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0231f-5419-41e5-8a94-9f79bb663e30 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:16.778433 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0231f-5419-41e5-8a94-9f79bb663e30 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:16.778546 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0231f-5419-41e5-8a94-9f79bb663e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:16.798782 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-ccd0231f-5419-41e5-8a94-9f79bb663e30 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:16.800046 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ccd0231f-5419-41e5-8a94-9f79bb663e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:16.878258 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1447/2875] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:27:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:16.889307 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62505e11-a26e-4300-b86a-6f90176a5e84 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:16.891174 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-62505e11-a26e-4300-b86a-6f90176a5e84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:16.954468 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62505e11-a26e-4300-b86a-6f90176a5e84 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:16.958712 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c641b-de81-44f6-b457-9e82a0f5005e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:16.959333 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c641b-de81-44f6-b457-9e82a0f5005e service neutron] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:16.959691 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c641b-de81-44f6-b457-9e82a0f5005e service neutron] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:16.959962 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c641b-de81-44f6-b457-9e82a0f5005e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:16.970675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c641b-de81-44f6-b457-9e82a0f5005e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:16.972119 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c641b-de81-44f6-b457-9e82a0f5005e service neutron] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:16.982480 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1429/2876] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:17.263523 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0231f-5419-41e5-8a94-9f79bb663e30 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:17.265784 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-ccd0231f-5419-41e5-8a94-9f79bb663e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:17.288254 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0231f-5419-41e5-8a94-9f79bb663e30 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:17.295891 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e213d2-ce76-441b-9dd2-ef2761a65ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:17.296917 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e213d2-ce76-441b-9dd2-ef2761a65ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:17.297242 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e213d2-ce76-441b-9dd2-ef2761a65ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:17.297553 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e213d2-ce76-441b-9dd2-ef2761a65ffd admin admin] PATH_INFO: `/v3/projects/758e37f4382744d1b4546345e431ee0a` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:17.298842 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e213d2-ce76-441b-9dd2-ef2761a65ffd admin admin] RBAC: Authorizing `identity:delete_project(project_id=758e37f4382744d1b4546345e431ee0a)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:17.301856 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e213d2-ce76-441b-9dd2-ef2761a65ffd admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:17.350011 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-26e213d2-ce76-441b-9dd2-ef2761a65ffd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '758e37f4382744d1b4546345e431ee0a', 'request_id': 'req-26e213d2-ce76-441b-9dd2-ef2761a65ffd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:17.358082 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-26e213d2-ce76-441b-9dd2-ef2761a65ffd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '758e37f4382744d1b4546345e431ee0a', 'request_id': 'req-26e213d2-ce76-441b-9dd2-ef2761a65ffd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:17.364862 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-26e213d2-ce76-441b-9dd2-ef2761a65ffd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '758e37f4382744d1b4546345e431ee0a', 'request_id': 'req-26e213d2-ce76-441b-9dd2-ef2761a65ffd'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:17.376267 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-26e213d2-ce76-441b-9dd2-ef2761a65ffd admin admin] The token cache is being invalidate because project 758e37f4382744d1b4546345e431ee0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:27:17.376601 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-26e213d2-ce76-441b-9dd2-ef2761a65ffd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:17.377252 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-26e213d2-ce76-441b-9dd2-ef2761a65ffd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:17.378774 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1448/2877] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:27:17 2022] DELETE /identity/v3/projects/758e37f4382744d1b4546345e431ee0a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:17.384221 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebad42f-dd10-4769-9843-93738b8d62dd None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:17.384542 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebad42f-dd10-4769-9843-93738b8d62dd None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:17.384851 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebad42f-dd10-4769-9843-93738b8d62dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:17.407884 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-4ebad42f-dd10-4769-9843-93738b8d62dd None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:17.409146 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-4ebad42f-dd10-4769-9843-93738b8d62dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:17.427835 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:17.474781 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1430/2878] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:27:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:27:17.483155 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e29a5d-2abe-48c2-acb9-f06c7fa0cc7d None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:17.483436 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e29a5d-2abe-48c2-acb9-f06c7fa0cc7d None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:17.483707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e29a5d-2abe-48c2-acb9-f06c7fa0cc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:17.504427 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-83e29a5d-2abe-48c2-acb9-f06c7fa0cc7d None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:17.505706 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-83e29a5d-2abe-48c2-acb9-f06c7fa0cc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:17.514440 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1449/2879] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:27:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:17.518702 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebad42f-dd10-4769-9843-93738b8d62dd None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:17.528633 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebad42f-dd10-4769-9843-93738b8d62dd None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:17.533225 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1c9e4-927b-4a21-8072-eb474ff53dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:17.533834 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1c9e4-927b-4a21-8072-eb474ff53dad admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:17.534115 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1c9e4-927b-4a21-8072-eb474ff53dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:17.534384 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1c9e4-927b-4a21-8072-eb474ff53dad admin admin] PATH_INFO: `/v3/domains` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:17.535067 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a1c9e4-927b-4a21-8072-eb474ff53dad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:27:17.535364 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a1c9e4-927b-4a21-8072-eb474ff53dad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:17.537482 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a1c9e4-927b-4a21-8072-eb474ff53dad admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:17.549095 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1431/2880] 10.222.0.93 () {58 vars in 1153 bytes} [Mon Aug 1 15:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:27:17.554235 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e29a5d-2abe-48c2-acb9-f06c7fa0cc7d None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:17.564382 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e29a5d-2abe-48c2-acb9-f06c7fa0cc7d None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:17.569200 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11542be1-ff0f-424a-9619-62748d7c0a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:17.569807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11542be1-ff0f-424a-9619-62748d7c0a6b admin admin] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:17.570094 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11542be1-ff0f-424a-9619-62748d7c0a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:17.570372 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11542be1-ff0f-424a-9619-62748d7c0a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:17.571040 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11542be1-ff0f-424a-9619-62748d7c0a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:17.572980 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11542be1-ff0f-424a-9619-62748d7c0a6b admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:17.607531 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1450/2881] 10.222.0.93 () {60 vars in 1153 bytes} [Mon Aug 1 15:27:17 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:27:17.611723 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1c9e4-927b-4a21-8072-eb474ff53dad admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:17.623950 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1c9e4-927b-4a21-8072-eb474ff53dad admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:17.680608 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabbf20-f3d4-4059-81ec-c6486f7a7539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:17.681363 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabbf20-f3d4-4059-81ec-c6486f7a7539 admin admin] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:17.681778 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabbf20-f3d4-4059-81ec-c6486f7a7539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:17.682164 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabbf20-f3d4-4059-81ec-c6486f7a7539 admin admin] PATH_INFO: `/v3/users` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:17.682948 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabbf20-f3d4-4059-81ec-c6486f7a7539 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:17.684993 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabbf20-f3d4-4059-81ec-c6486f7a7539 admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:17.706796 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1432/2882] 10.222.0.93 () {60 vars in 1144 bytes} [Mon Aug 1 15:27:17 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:27:17.711340 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11542be1-ff0f-424a-9619-62748d7c0a6b admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:17.722012 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11542be1-ff0f-424a-9619-62748d7c0a6b admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:17.727565 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606d4f6-f2b9-4147-8a86-a4528dbd75a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:17.728213 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606d4f6-f2b9-4147-8a86-a4528dbd75a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:17.728533 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606d4f6-f2b9-4147-8a86-a4528dbd75a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:17.728807 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606d4f6-f2b9-4147-8a86-a4528dbd75a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:17.729500 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606d4f6-f2b9-4147-8a86-a4528dbd75a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:17.731129 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606d4f6-f2b9-4147-8a86-a4528dbd75a7 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:17.742044 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1451/2883] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:27:17.746165 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabbf20-f3d4-4059-81ec-c6486f7a7539 admin admin] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:17.756281 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabbf20-f3d4-4059-81ec-c6486f7a7539 admin admin] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:17.761073 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a4c83-125a-4cc6-9a68-b8458c2756ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:17.761684 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a4c83-125a-4cc6-9a68-b8458c2756ec admin admin] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:17.761978 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a4c83-125a-4cc6-9a68-b8458c2756ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:17.762250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a4c83-125a-4cc6-9a68-b8458c2756ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:17.762980 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a4c83-125a-4cc6-9a68-b8458c2756ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:17.764586 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a4c83-125a-4cc6-9a68-b8458c2756ec admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:17.776344 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1433/2884] 10.222.0.93 () {58 vars in 1122 bytes} [Mon Aug 1 15:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:17.780897 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606d4f6-f2b9-4147-8a86-a4528dbd75a7 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:17.791414 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606d4f6-f2b9-4147-8a86-a4528dbd75a7 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:17.795983 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a042c-dbd9-4f13-a3ae-8c607f2f1e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:17.796668 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a042c-dbd9-4f13-a3ae-8c607f2f1e21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:17.796951 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a042c-dbd9-4f13-a3ae-8c607f2f1e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:17.797224 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a042c-dbd9-4f13-a3ae-8c607f2f1e21 admin admin] PATH_INFO: `/v3/projects/7e9b91391b71463abc4cb8ecb75ab566/users/08a80c6e01b34deeaf642fe1c4309be1/roles/c65b8ad9ade94760af715b3899bdd799` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:17.815541 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a042c-dbd9-4f13-a3ae-8c607f2f1e21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e9b91391b71463abc4cb8ecb75ab566, user_id=08a80c6e01b34deeaf642fe1c4309be1, role_id=c65b8ad9ade94760af715b3899bdd799)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:17.818393 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a042c-dbd9-4f13-a3ae-8c607f2f1e21 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:17.831653 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1452/2885] 10.222.0.93 () {60 vars in 1483 bytes} [Mon Aug 1 15:27:17 2022] PUT /identity/v3/projects/7e9b91391b71463abc4cb8ecb75ab566/users/08a80c6e01b34deeaf642fe1c4309be1/roles/c65b8ad9ade94760af715b3899bdd799 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:17.844739 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c7229-33a4-4ab0-b8cb-70acdfbb5c85 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:17.845028 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c7229-33a4-4ab0-b8cb-70acdfbb5c85 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:17.845331 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c7229-33a4-4ab0-b8cb-70acdfbb5c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:17.875100 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-cf6c7229-33a4-4ab0-b8cb-70acdfbb5c85 None None] MFA Rules not processed for user `08a80c6e01b34deeaf642fe1c4309be1`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:17.876255 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-cf6c7229-33a4-4ab0-b8cb-70acdfbb5c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:17.943543 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1434/2886] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:27:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:17.954389 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a042c-dbd9-4f13-a3ae-8c607f2f1e21 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:17.956163 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-037a042c-dbd9-4f13-a3ae-8c607f2f1e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:17.979522 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:18.015443 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a042c-dbd9-4f13-a3ae-8c607f2f1e21 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:18.019143 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8529c1e-fe37-4ad6-bb72-c0038e4af66b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67bf3011e2bd47a3b5cf5e0b1173f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:18.019750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8529c1e-fe37-4ad6-bb72-c0038e4af66b service cinder] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:18.020560 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8529c1e-fe37-4ad6-bb72-c0038e4af66b service cinder] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:18.020828 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8529c1e-fe37-4ad6-bb72-c0038e4af66b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:18.029251 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8529c1e-fe37-4ad6-bb72-c0038e4af66b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:18.030677 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8529c1e-fe37-4ad6-bb72-c0038e4af66b service cinder] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:18.040441 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1453/2887] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:27:19.414271 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2c808-57b5-4b31-9246-fb1b5caa239d None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:19.414831 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2c808-57b5-4b31-9246-fb1b5caa239d None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:19.414831 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2c808-57b5-4b31-9246-fb1b5caa239d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:19.436220 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-fed2c808-57b5-4b31-9246-fb1b5caa239d None None] MFA Rules not processed for user `08a80c6e01b34deeaf642fe1c4309be1`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:19.437722 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-fed2c808-57b5-4b31-9246-fb1b5caa239d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:19.445224 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1435/2888] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:27:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:28.036350 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=17 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:29.430247 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:33.472449 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782aee6-1061-437d-b49e-61a3d25609c8 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:33.473086 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782aee6-1061-437d-b49e-61a3d25609c8 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:33.473086 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782aee6-1061-437d-b49e-61a3d25609c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:33.530202 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-3782aee6-1061-437d-b49e-61a3d25609c8 None None] MFA Rules not processed for user `4370f8f474c2446e989d0d8c5b75e87a`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:33.531453 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3782aee6-1061-437d-b49e-61a3d25609c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:33.602447 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1454/2889] 10.222.0.93 () {58 vars in 1009 bytes} [Mon Aug 1 15:27:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:33.658564 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2c808-57b5-4b31-9246-fb1b5caa239d None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:33.661167 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-fed2c808-57b5-4b31-9246-fb1b5caa239d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:33.807964 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2c808-57b5-4b31-9246-fb1b5caa239d None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:33.813301 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb0c8a-f913-4e53-b544-dba83b96d372 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4df37b02152c48509243b73ebbf49d8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:33.813924 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb0c8a-f913-4e53-b544-dba83b96d372 service swift] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:33.814199 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb0c8a-f913-4e53-b544-dba83b96d372 service swift] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:33.814470 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb0c8a-f913-4e53-b544-dba83b96d372 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:33.823912 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb0c8a-f913-4e53-b544-dba83b96d372 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:33.825549 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb0c8a-f913-4e53-b544-dba83b96d372 service swift] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:33.834319 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1436/2890] 10.222.0.93 () {60 vars in 1384 bytes} [Mon Aug 1 15:27:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:27:34.399220 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f820a-7296-47bc-adf3-3bfe5a8a6c43 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:34.399647 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f820a-7296-47bc-adf3-3bfe5a8a6c43 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:34.399969 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f820a-7296-47bc-adf3-3bfe5a8a6c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:34.419989 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-8e9f820a-7296-47bc-adf3-3bfe5a8a6c43 None None] MFA Rules not processed for user `4370f8f474c2446e989d0d8c5b75e87a`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:34.421347 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-8e9f820a-7296-47bc-adf3-3bfe5a8a6c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:34.429280 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1455/2891] 10.222.0.93 () {58 vars in 967 bytes} [Mon Aug 1 15:27:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:34.443250 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb0c8a-f913-4e53-b544-dba83b96d372 service swift] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:34.454989 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb0c8a-f913-4e53-b544-dba83b96d372 service swift] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:34.460522 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80319eb6-d5a1-4f71-968a-145331902711 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4df37b02152c48509243b73ebbf49d8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:34.461148 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80319eb6-d5a1-4f71-968a-145331902711 service swift] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:34.461429 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80319eb6-d5a1-4f71-968a-145331902711 service swift] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:34.461706 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80319eb6-d5a1-4f71-968a-145331902711 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:34.471017 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80319eb6-d5a1-4f71-968a-145331902711 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:34.472676 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80319eb6-d5a1-4f71-968a-145331902711 service swift] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:34.482511 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1437/2892] 10.222.0.93 () {60 vars in 1384 bytes} [Mon Aug 1 15:27:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:27:34.717228 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda39a25-a5cb-45a5-8a2a-692e093cd067 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:34.717777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda39a25-a5cb-45a5-8a2a-692e093cd067 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:34.717777 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda39a25-a5cb-45a5-8a2a-692e093cd067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:34.738594 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-cda39a25-a5cb-45a5-8a2a-692e093cd067 None None] MFA Rules not processed for user `4370f8f474c2446e989d0d8c5b75e87a`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:34.740386 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-cda39a25-a5cb-45a5-8a2a-692e093cd067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:34.748041 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1456/2893] 10.222.0.93 () {58 vars in 1009 bytes} [Mon Aug 1 15:27:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:34.760751 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80319eb6-d5a1-4f71-968a-145331902711 service swift] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:34.774151 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80319eb6-d5a1-4f71-968a-145331902711 service swift] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:34.779696 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71b35f-8ede-4f21-b152-6f43848fcb6f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4df37b02152c48509243b73ebbf49d8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:34.780343 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71b35f-8ede-4f21-b152-6f43848fcb6f service swift] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:34.780654 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71b35f-8ede-4f21-b152-6f43848fcb6f service swift] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:34.780929 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71b35f-8ede-4f21-b152-6f43848fcb6f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:34.792034 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71b35f-8ede-4f21-b152-6f43848fcb6f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:34.793731 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71b35f-8ede-4f21-b152-6f43848fcb6f service swift] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:34.804414 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1438/2894] 10.222.0.93 () {60 vars in 1384 bytes} [Mon Aug 1 15:27:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:27:35.193715 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e9c46-8725-4682-a172-b6edc04f1aa3 None None] REQUEST_METHOD: `POST` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:35.194002 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e9c46-8725-4682-a172-b6edc04f1aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:35.194282 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e9c46-8725-4682-a172-b6edc04f1aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:35.214963 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.auth.core [None req-3b1e9c46-8725-4682-a172-b6edc04f1aa3 None None] MFA Rules not processed for user `4370f8f474c2446e989d0d8c5b75e87a`. Rule list: `[]` (Enabled: `True`). {{(pid=80412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:35.216166 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3b1e9c46-8725-4682-a172-b6edc04f1aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:35.223943 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1457/2895] 10.222.0.93 () {58 vars in 967 bytes} [Mon Aug 1 15:27:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:35.239135 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71b35f-8ede-4f21-b152-6f43848fcb6f service swift] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:35.251708 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71b35f-8ede-4f21-b152-6f43848fcb6f service swift] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:35.257675 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee1236d-6e05-4f4b-83c7-89f57cc482b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4df37b02152c48509243b73ebbf49d8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:35.258380 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1236d-6e05-4f4b-83c7-89f57cc482b9 service swift] REQUEST_METHOD: `GET` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:35.258766 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1236d-6e05-4f4b-83c7-89f57cc482b9 service swift] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:35.259129 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1236d-6e05-4f4b-83c7-89f57cc482b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:35.268488 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee1236d-6e05-4f4b-83c7-89f57cc482b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:35.270623 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee1236d-6e05-4f4b-83c7-89f57cc482b9 service swift] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:35.279560 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1439/2896] 10.222.0.93 () {60 vars in 1384 bytes} [Mon Aug 1 15:27:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:27:37.609004 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e9c46-8725-4682-a172-b6edc04f1aa3 None None] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:37.682678 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e9c46-8725-4682-a172-b6edc04f1aa3 None None] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:37.687612 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe0e12-89ea-4ef9-993a-0a11e33c5161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:37.688277 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe0e12-89ea-4ef9-993a-0a11e33c5161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:37.688602 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe0e12-89ea-4ef9-993a-0a11e33c5161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:37.688884 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe0e12-89ea-4ef9-993a-0a11e33c5161 admin admin] PATH_INFO: `/v3/users/08a80c6e01b34deeaf642fe1c4309be1` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:37.690157 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe0e12-89ea-4ef9-993a-0a11e33c5161 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08a80c6e01b34deeaf642fe1c4309be1)` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:37.692750 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe0e12-89ea-4ef9-993a-0a11e33c5161 admin admin] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:37.752331 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-3dbe0e12-89ea-4ef9-993a-0a11e33c5161 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08a80c6e01b34deeaf642fe1c4309be1'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:37.759134 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:37.761524 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-3dbe0e12-89ea-4ef9-993a-0a11e33c5161 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08a80c6e01b34deeaf642fe1c4309be1'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:37.775365 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-3dbe0e12-89ea-4ef9-993a-0a11e33c5161 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08a80c6e01b34deeaf642fe1c4309be1'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:37.776298 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-3dbe0e12-89ea-4ef9-993a-0a11e33c5161 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08a80c6e01b34deeaf642fe1c4309be1'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:37.777150 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.notifications [None req-3dbe0e12-89ea-4ef9-993a-0a11e33c5161 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08a80c6e01b34deeaf642fe1c4309be1'} {{(pid=80412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:37.797436 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1458/2897] 10.222.0.93 () {58 vars in 1224 bytes} [Mon Aug 1 15:27:37 2022] DELETE /identity/v3/users/08a80c6e01b34deeaf642fe1c4309be1 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:37.803160 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a165957-1c25-4024-88e3-39e11c2279c7 None None] REQUEST_METHOD: `POST` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:37.803727 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a165957-1c25-4024-88e3-39e11c2279c7 None None] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:37.804158 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a165957-1c25-4024-88e3-39e11c2279c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:37.839501 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.auth.core [None req-1a165957-1c25-4024-88e3-39e11c2279c7 None None] MFA Rules not processed for user `b035f76541b24134b8802d5327d7615c`. Rule list: `[]` (Enabled: `True`). {{(pid=80411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:37.842525 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-1a165957-1c25-4024-88e3-39e11c2279c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:37.927211 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1440/2898] 10.222.0.93 () {58 vars in 958 bytes} [Mon Aug 1 15:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:37.937607 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe0e12-89ea-4ef9-993a-0a11e33c5161 admin admin] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:37.939328 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-3dbe0e12-89ea-4ef9-993a-0a11e33c5161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:38.006311 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe0e12-89ea-4ef9-993a-0a11e33c5161 admin admin] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:38.010707 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f402f-40b3-4ea2-80e8-fb3dee7811d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eba5f64833b482d81c53721c8442970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:38.011343 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f402f-40b3-4ea2-80e8-fb3dee7811d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:38.011623 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f402f-40b3-4ea2-80e8-fb3dee7811d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:38.011896 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f402f-40b3-4ea2-80e8-fb3dee7811d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:38.023152 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f402f-40b3-4ea2-80e8-fb3dee7811d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:38.024755 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f402f-40b3-4ea2-80e8-fb3dee7811d9 service neutron] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:38.036635 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1459/2899] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:38.234828 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a165957-1c25-4024-88e3-39e11c2279c7 None None] Authenticating user token {{(pid=80411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:38.236508 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.fernet_utils [None req-1a165957-1c25-4024-88e3-39e11c2279c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:38.251895 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a165957-1c25-4024-88e3-39e11c2279c7 None None] Validating token access rules against request {{(pid=80411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:38.261094 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f3ee3-f090-48eb-919c-1ebb59f617af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b035f76541b24134b8802d5327d7615c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ee187de5744709bb65431943eb0200', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:38.261786 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f3ee3-f090-48eb-919c-1ebb59f617af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:38.262228 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f3ee3-f090-48eb-919c-1ebb59f617af admin admin] SCRIPT_NAME: `/identity` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:38.262494 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f3ee3-f090-48eb-919c-1ebb59f617af admin admin] PATH_INFO: `/v3/projects/7e9b91391b71463abc4cb8ecb75ab566` {{(pid=80411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:38.263971 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f3ee3-f090-48eb-919c-1ebb59f617af admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e9b91391b71463abc4cb8ecb75ab566)` {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:38.266088 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f3ee3-f090-48eb-919c-1ebb59f617af admin admin] RBAC: Authorization granted {{(pid=80411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:38.313074 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:38.334598 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-bf6f3ee3-f090-48eb-919c-1ebb59f617af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e9b91391b71463abc4cb8ecb75ab566', 'request_id': 'req-bf6f3ee3-f090-48eb-919c-1ebb59f617af'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:38.346051 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-bf6f3ee3-f090-48eb-919c-1ebb59f617af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e9b91391b71463abc4cb8ecb75ab566', 'request_id': 'req-bf6f3ee3-f090-48eb-919c-1ebb59f617af'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:38.353745 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-bf6f3ee3-f090-48eb-919c-1ebb59f617af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e9b91391b71463abc4cb8ecb75ab566', 'request_id': 'req-bf6f3ee3-f090-48eb-919c-1ebb59f617af'} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:38.371633 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-bf6f3ee3-f090-48eb-919c-1ebb59f617af admin admin] The token cache is being invalidate because project 7e9b91391b71463abc4cb8ecb75ab566 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:27:38.371935 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-bf6f3ee3-f090-48eb-919c-1ebb59f617af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:38.372680 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG keystone.notifications [None req-bf6f3ee3-f090-48eb-919c-1ebb59f617af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:38.374288 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: [pid: 80411|app: 0|req: 1441/2900] 10.222.0.93 () {58 vars in 1233 bytes} [Mon Aug 1 15:27:38 2022] DELETE /identity/v3/projects/7e9b91391b71463abc4cb8ecb75ab566 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:48.031484 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=35,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:48.367201 dv-cinder-803823-14-iscsi devstack@keystone.service[80411]: DEBUG dbcounter [-] [80411] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:32.679855 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f402f-40b3-4ea2-80e8-fb3dee7811d9 service neutron] Authenticating user token {{(pid=80412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:32.681481 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-fa1f402f-40b3-4ea2-80e8-fb3dee7811d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:32.834151 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f402f-40b3-4ea2-80e8-fb3dee7811d9 service neutron] Validating token access rules against request {{(pid=80412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:32.902924 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524947e1-320b-4cc5-956c-4e68d0a448a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07e3e25e6a8466fb237dcb836aecda2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac9f7c74597642b0acecc956429ac4d8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:32.903552 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524947e1-320b-4cc5-956c-4e68d0a448a9 service placement] REQUEST_METHOD: `GET` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:32.904392 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524947e1-320b-4cc5-956c-4e68d0a448a9 service placement] SCRIPT_NAME: `/identity` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:32.904723 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524947e1-320b-4cc5-956c-4e68d0a448a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:32.906495 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.fernet_utils [None req-524947e1-320b-4cc5-956c-4e68d0a448a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:32.919099 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524947e1-320b-4cc5-956c-4e68d0a448a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:32.920789 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524947e1-320b-4cc5-956c-4e68d0a448a9 service placement] RBAC: Authorization granted {{(pid=80412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:32.932008 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: [pid: 80412|app: 0|req: 1460/2901] 10.222.0.93 () {60 vars in 1365 bytes} [Mon Aug 1 15:30:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:30:42.927420 dv-cinder-803823-14-iscsi devstack@keystone.service[80412]: DEBUG dbcounter [-] [80412] Writing DB stats keystone:SELECT=89 {{(pid=80412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}